Biology of Neural Networks

The Synapse - Where the Magic Actually Happens

In the first blog, we met the neuron. The tiny decision maker. The LEGO brick of intelligence. Unimpressive alone, unstoppable together.

But here is a question that most people don't stop to think about. How do neurons actually talk to each other? What is the mechanism? What is the connection?

In your brain, there is a tiny gap between every pair of connected neurons. This gap is called the synapse. One neuron releases chemicals across this gap. The other neuron catches those chemicals. That is how a message travels.

In an artificial neural network, there are no chemicals. There is no gap. There is just a single number. That number is called the weight.

And here is the honest truth. The neuron gets all the fame. The neuron gets all the attention. But the weight is where the intelligence actually lives. The weight is the real hero.

The Synapse in Your Brain - A Closer Look

Two neurons in your brain never actually touch each other. There is always a microscopic gap between them. This gap is the synapse.

When Neuron A wants to talk to Neuron B, here is what happens:

Neuron A fires. An electrical signal travels down its axon.
At the end of the axon, the signal triggers the release of chemicals called neurotransmitters.
These chemicals float across the tiny gap to Neuron B's dendrites.
Neuron B catches the chemicals and decides: "Okay, someone is talking to me. Should I fire or not?"

This whole process takes about a millisecond. That's fast. But not as fast as what happens in a computer.

But here is the most important part. The strength of the connection between Neuron A and Neuron B is not fixed. It changes over time. The more they talk to each other, the stronger the connection becomes.

This is called Hebbian plasticity, named after the psychologist Donald Hebb who proposed it in 1949. He summed it up in a famous phrase:

"Neurons that fire together, wire together."

Jab do neurons baar baar ek saath fire karte hain, toh unka aapas ka rishta strong ho jaata hai. Jaise do dost jo saath saath bade hue hain. Unki understanding aisi hoti hai ki ek baat kare toh doosra samajh jaaye. Bina puche.

The opposite is also true. If two neurons stop firing together, their connection weakens. Use it or lose it.

This is how your brain learns. Every time you practice the guitar, the connections between the neurons controlling your fingers get stronger. Eventually, you don't have to think about where to put your fingers. The neurons just know.

The Synapse in Artificial Neural Networks

In an artificial neural network, the synapse is represented by one simple thing. A number. The weight.

Every connection between every pair of artificial neurons has its own weight. A number that represents how strong that connection is.

High positive weight (like 5.2): Strong excitatory connection. If the first neuron fires, the second one definitely hears it. The signal gets amplified.
Low weight (like 0.1): Weak connection. The signal barely gets through. It's like whispering in a noisy room.
Negative weight (like -3.7): Inhibitory connection. This neuron actively tells the next one "calm down. don't fire." It reduces the signal instead of amplifying it.
Zero weight (like 0.0): No connection at all. These neurons might as well not exist to each other.

A typical small neural network has millions of these weights. A large one like GPT-4 has about 1.76 trillion weights.

And here is the most remarkable thing. Every single one of those numbers was learned from data. No human went in and said "this weight should be 0.42." The model figured it out by itself during training.

Bada sochiye. 1.76 trillion tiny numbers. Har ek number ek cheez represent karta hai jo model ne data se seekha hai. "Jab yeh pixel aise dikhe toh dhyaan do." "Jab yeh word uss word ke baad aaye toh iska matlab yeh hai." Trillions of tiny pieces of knowledge, all working together.

How Weights Change During Training

Before training starts, all the weights in a neural network are random. Just random numbers. The model knows nothing.

When you first show a randomly initialized model a picture of a cat, it has no idea what it's looking at. It might predict "toaster" with 10% confidence. It might predict "car" with 15% confidence. It's basically guessing.

Then something amazing happens. You tell the model: "Wrong. That was a cat."

The model calculates the error. How wrong was it? Then it traces that error back through every single weight and asks a question:

"Mr. Weight number 492,301. How much did you contribute to this mistake?"

And every weight gets adjusted. Just a tiny bit. 0.001 here. -0.0005 there. Almost invisible changes.

Then you show it another cat. And another. And another. Thousand of cats. Millions of cats. Every time, the weights adjust a tiny bit more.

Eventually, the weights converge to the right values. The model now knows what a cat looks like.

Yeh waise hi hai jaise baccha seekhta hai. Pehli baar jab baccha garm chulhe ko chhuta hai, toh jal jaata hai. Uske dimaag mein ek weight adjust ho jaata hai: "Chulha = Garm = Danger." Agli baar, woh nahi chhuega.

The Chemical Synapse vs The Digital Weight

Let's put biology and AI side by side so you can see the similarities and differences:

Biological Synapse

Transmits via chemicals (neurotransmitters)
Takes about 1 millisecond per transmission
Strengthens based on frequency of use (Hebbian learning)
Can die or weaken with age or disuse
Cannot be copied or duplicated
Energy efficient (each firing uses tiny amount of energy)
Can rewire itself to form new connections

Artificial Weight

Transmits via multiplication (just math)
Takes less than a nanosecond per operation
Updates based on backpropagation and error calculation
Stays forever unless deliberately changed
Can be copied infinitely (duplicate the model file)
Energy hungry (each operation needs a computer chip)
Fixed architecture during training

The artificial version is faster, more durable, and perfectly copyable. But the biological version is more flexible, more efficient, and can rewire itself in ways we still don't fully understand.

Nature wins on efficiency. Silicon wins on speed.

A Real World Analogy: The WhatsApp Group

Imagine you're in a WhatsApp group with 200 people. You don't care equally about all of them. Some people matter a lot. Some people you ignore completely. Some people actively annoy you.

Let's map this to weights:

Your best friend has a high positive weight. Every message from them, you read immediately. You reply. You care. Their opinion matters.
A distant cousin has a low weight. When they message, you glance at it. Maybe reply later. Maybe not. Doesn't move the needle.
Your toxic ex who somehow ended up in the group? That's a negative weight. You actively mute their messages. You ignore them. Their presence reduces your interest in the group.
A random person you don't know has a weight of zero. You don't even notice their messages.

Now imagine your brain is like this group. Every neuron is a person. Every weight is how much one person listens to another person. And learning? Learning is just adjusting these weights based on experience.

When you have a bad experience with someone, their weight goes down (becomes more negative). When someone helps you, their weight goes up. Your entire personality, your knowledge, your skills, your relationships. All of it is just weights between neurons.

The Most Beautiful Part

Here is the deepest thought in this entire blog.

Every weight in a neural network represents a tiny hypothesis about the world.

"I have learned that when I see this specific pattern of pixels, it's probably a cat."

"I have learned that when this word appears next to that word, the sentence is probably positive."

"I have learned that this combination of inputs is important, and that combination is noise."

Each weight is a tiny piece of learned knowledge. Nothing more. But put a trillion of them together, and you have a model that can write poetry, diagnose diseases, compose music, and have conversations that feel almost human.

We have not programmed intelligence. We have created a system that learns its own intelligence, one weight at a time.

The neuron is the stage. But the weight is the actor. And the play? The play is everything the network has ever learned. Every weight is a memory. Every connection is a lesson. Every number tells a story.