Two Jupyter notebooks that implement and visualize Hopfield networks (a classical recurrent associative memory architecture).
1Store
Patterns are written into the weight matrix with Hebb's rule.
2Corrupt
A noisy or incomplete version of a stored pattern is presented.
3Update
Neurons update one at a time, and each update lowers the network's energy.
4Settle
The network stops in the nearest stored memory. The notebook animates this frame by frame.
Retrieval is the loop in the middle: the network is not looking anything up, it is falling downhill on an energy surface until it stops moving. That is the part the advanced notebook animates frame by frame.
What’s in the notebooks
Basic (2-neuron): Implements a minimal Hopfield network with 2 neurons to show the core concept: energy landscape, weight matrix from Hebb’s rule, and convergence to a stored pattern. Visualizes the energy function as a 3D surface to make the “energy minimization = memory retrieval” idea concrete.
Advanced (5×5 grid): Scales to a 25-neuron network arranged as a 5×5 image grid. Demonstrates:
- Storing multiple binary patterns (5×5 pixel “images”)
- Presenting a noisy or incomplete version of a stored pattern
- Watching the network iterate via asynchronous update rules until it converges to the nearest stored memory
- Animating the convergence step-by-step with Matplotlib
Why Hopfield networks
Hopfield networks predate modern deep learning and aren’t used in production ML today, but they make the core ideas of energy-based models, associative memory, and recurrent dynamics unusually visible. The energy landscape is something you can actually plot. The weight update rule is one equation. It’s a good starting point for understanding what “the network has learned something” means at a mathematical level before moving to neural networks where everything is less interpretable.
Stack
- Python 3, NumPy, Matplotlib
- Jupyter Notebook
Timeline: Oct 2024 · Research notebook