When studying, we redline important information (strengthening) and forget unimportant information (weakening). The same goes for the synapses in our brains. So, what exactly do synapses use as a criterion to strengthen or weaken connections? The secret rule discovered by neuroscientists is STDP (Spike-Timing-Dependent Plasticity).
In this article, we'll explore the principles of implementing this complex biological learning rule in hardware using RRAM devices. This is the foundation of the Spiking Neural Network (SNN).
1. Hebbian Learning
To understand STDP, you must first understand ‘Hebb’s Rule’.
“Cells that fire together, wire together.”
That is, when the cause (pre-synapse, input) and the result (post-synapse, output) are activated at similar times, the brain judges, “Oh, these two are related!” and strengthens the weight.
STDP adds a sophisticated concept called ‘Timing’ to this.
2. What is STDP?: Timing is everything.
STDP adjusts synaptic weights based on the order in which signals come in, i.e., Δt (time difference).
① LTP (Long-Term Potentiation): Δt > 0
- Situation: The output signal (Post) occurs immediately after the input signal (Pre).
- Analysis: “The input causes the output!” (Causality established)
- Result: Strengthens synaptic connections (reduces component resistance)
② LTD (Long-Term Depression): Δt < 0
- Situation: The output signal (Post) goes out first, and the input signal (Pre) comes in later .
- Analysis: "This input has nothing to do with the result?" (No causal relationship)
- Result: Disconnects synaptic connections (increases device resistance)
3. Implementing STDP with RRAM: Waveform Overlapping Technique
RRAM is a two-terminal device. It doesn't have a complex timing calculator like the brain. So how does it detect who arrived first and adjust its resistance? The secret lies in the pulse shape (waveform shaping).
Operation mechanism (Superposition)
The actual voltage across the device (Vnet) is the difference between the upper electrode voltage (Vtop) and the lower electrode voltage (Vbottom).
- When the time difference is appropriate (Pre -> Post):
- When two pulses overlap, the voltage difference exceeds the set voltage (Vset). -> The element resistance decreases (LTP).
- When the time difference is reversed (Post -> Pre):
- When two pulses overlap, the voltage difference exceeds the Reset Voltage (Vreset). -> Increase in element resistance (LTD).
- When the time difference is too large:
- The voltage is low because the pulses do not meet each other -> no change in element resistance.
With just these simple laws of physics, RRAM learns on its own without any separate CPU computation.
4. Why is it important? (Unsupervised Learning)
Conventional deep learning (DNN) is supervised learning, which provides the correct answer (label) and calculates and updates the error. This requires enormous computational effort and consumes significant power.
However, RRAM (SNN) unsupervised learning with STDP is possible.
- Even without being told the correct answer, it automatically strengthens the connection between “signals that frequently come in together” by simply looking at the temporal patterns of the data.
- This is most similar to how the human brain learns about the world as a baby, and is a key technology for implementing ultra-low-power edge AI .
5. Conclusion: The most brain-like memory
The real reason RRAM is being called the next-generation AI device isn't simply because it has analog resistance. It's because it offers the most elegant and simple hardware implementation of the STDP learning rule, which adjusts connection strength solely by pulse timing.
As RRAM research deepens, we will begin to blur the lines between studying semiconductors and studying the brain. Neuromorphic engineeringlies at the very edge of this divide.
References: Nanoscale Memristor Device as Synapse in Neuromorphic Systems
