This is an experiment based on Peter Foldiak's work on developing sparse codes. We can take a single layer of neurons (perceptrons) and add anti-Hebbian feedback connections between them that can learn to code patterns in such a way that statistical dependency between the elements of the representation is reduced while preserving information. Even this simple network is shown to learn the independent patterns of the dataset, providing a simple alternative to PCA.

Project link: https://github.com/derekwan1/sparse-bars

Derek Wan

Software Engineer and Teacher

141

3