Forward-Forward

Test the performance of the Forward-Forward algorithm in PyTorch.

Training
PyTorch

This module implements a complete open-source version of Geoffrey Hinton's Forward Forward Algorithm, an alternative approach to backpropagation.

The Forward Forward algorithm is a method for training deep neural networks that replaces the backpropagation forward and backward passes with two forward passes, one with positive (i.e., real) data and the other with negative data that could be generated by the network itself.

Unlike the backpropagation approach, Forward-Forward does not require calculating the gradient of the loss function with respect to the network parameters. Instead, each optimization step can be performed locally and the weights of each layer can be updated immediately after the layer has performed its forward pass.

Try it out on GitHub, and reach out if you have any feedback!

Learn more about Forward-Forward