Biology of Neural Networks

The Neuron - The Smallest Unit of Intelligence

Your brain has about 86 billion neurons. 86 billion. That is more than the number of people who have ever lived on this planet. And every single one of those 86 billion tiny cells is packed inside a 3 pound blob sitting inside your skull.

Every thought you have ever had. Every emotion. Every memory. Every dream. Every time you felt butterflies in your stomach or got goosebumps from a song. All of it. Just neurons talking to each other.

Now imagine taking that idea and copying it into a computer. Instead of biological cells, you use math. Instead of chemicals, you use numbers. Instead of a brain, you build an artificial neural network.

And at the very bottom of it all, the smallest piece, the foundation of everything, is one tiny thing. A single neuron.

Let's Start with the Real One

A biological neuron looks like a weird little alien tentacle. It has three main parts:

1. Dendrites (The Receivers) These are like little branches sticking out of the neuron. Think of them as antennas. They catch signals coming from other neurons. A single neuron can have thousands of dendrites, like a satellite dish array, constantly listening for messages.

2. Cell Body or Soma (The Decision Maker) This is the main part of the neuron. All the signals from the dendrites come here and get added up. If the total signal is strong enough, the neuron decides to fire. If it's too weak, it stays quiet. Simple as that.

3. Axon (The Transmitter) If the neuron decides to fire, the signal travels down the axon to the next neurons. The axon is like a wire that carries the message to its destination. And at the end of the axon, it branches out to connect with thousands of other neurons.

Baaten karne ka tareeka aisa hai jaise ek bada sa office ho jahan har kisi ke paas walkie talkie ho. Har koi sun raha hai, lekin kuch messages hi aage bheje jaate hain.

A neuron in your brain is basically a very small, very fast, very efficient "should I tell the next guy?" machine. That is all it does. Receive signals. Add them up. Decide. Fire or don't fire.

Now the Artificial One

An artificial neuron is a copy of this exact idea, except it runs on math instead of biology.

1. Inputs (The Dendrites) Instead of chemical signals, we have numbers. These numbers could be anything. Pixel values from an image. Word counts from a sentence. Sensor readings from a robot. Stock prices from the market. Any data that can be turned into a number goes here.

2. Weighted Sum + Activation Function (The Cell Body) Every input signal gets multiplied by a number called a weight. Some inputs matter more, so they get higher weights. Some inputs are irrelevant, so their weights are low or even negative.

Then all these weighted inputs get added up, plus a little extra number called a bias (think of it as the neuron's mood, its default tendency to fire or not).

Then the total goes through an activation function. This is the decision maker. It decides whether the final signal is strong enough to pass forward.

3. Output (The Axon) A single number comes out. Either the neuron fires (output is something meaningful) or it stays quiet (output is zero or close to zero).

And that is it. A single artificial neuron is just a math equation. Nothing more. One formula. One number out. Not impressive at all by itself.

Arey bhai, ek equation se kya hi hoga? Lekin 86 billion ka jhund banao, toh kamaal ho jaata hai.

The Perceptron - The Grandfather of All Neurons

In 1958, a psychologist named Frank Rosenblatt did something that would change the world forever. He built the first artificial neuron and called it the Perceptron.

The Perceptron was a simple device. You gave it inputs, it multiplied them by weights, added a bias, and checked if the result crossed a threshold. If yes, it output 1. If no, it output 0.

Simple, right?

But here is the amazing part. The Perceptron could learn. You could train it to classify things. Show it pictures of cats and dogs, adjust the weights a little, and eventually it could tell them apart. Not perfectly. But it could learn.

The problem? The Perceptron could only learn things that were linearly separable. Fancy term, but the meaning is simple. It could only draw straight lines. It could separate two categories if there was a clean straight line between them.

Aur duniya mein straight lines se kaam nahi chalta. Relationships are messy. Data is messy. Life is messy. The Perceptron hit a wall.

For years, people thought neural networks were a dead end. And then someone figured out how to stack neurons on top of each other, and everything changed.

The Perceptron walked so Deep Learning could run. And then it tripped on a non-linear problem. But that's okay. We fixed it.

Why One Neuron Is Completely Useless

One neuron is like one person trying to move a sofa up three flights of stairs. Technically possible? Maybe. Practically a disaster? Absolutely.

A single neuron can only make one simple decision. "Input A plus Input B minus Input C. Is the total more than 0.5? Yes or no?"

Not very useful.

But here is the key insight that changed everything. Put 86 billion of them together, wire them up in the right way, and now you have a brain. A brain that can compose a symphony, fall in love, build a rocket ship, and wonder about the meaning of the universe.

Put a few million artificial neurons together, wire them up, train them on enough data, and now you have a network that can:

Recognize your face when you unlock your phone
Understand your voice when you say "Hey Siri, what's the weather?"
Translate any language into any other language
Beat the world champion at the most complex board game ever invented
Generate paintings that sell for millions of dollars

The magic isn't the neuron. The magic is the network. The connections. The interactions. The emergent behavior that comes from putting simple things together in clever ways.

Ek cheez akeli mehenga nahi hoti. Lekin 86 billion milkar kamaal kar dete hain. Same with neurons.

A Detailed Example: The "Should I Go Out?" Neuron

Let's build a single neuron in our head. Imagine a neuron that decides whether you should leave the house today.

Inputs to this neuron:

Input 1: Is it raining outside? (1 = yes, 0 = no)
Input 2: Is your phone battery charged? (1 = yes, 0 = no)
Input 3: Do you have money in your pocket? (1 = yes, 0 = no)
Input 4: Are your friends available? (1 = yes, 0 = no)
Input 5: Do you feel lazy today? (1 = yes, 0 = no)

Weights on these inputs (how much each one matters):

Weight 1: 0.6 (Rain is annoying, but not the end of the world)
Weight 2: 0.9 (Dead phone is basically an emergency these days)
Weight 3: 0.7 (No money means no fun)
Weight 4: 0.8 (Friends make everything better)
Weight 5: -0.9 (Laziness is strong, it actively works against going out)

Bias: -0.3 (You are slightly lazy by default. The neuron needs a push to fire.)

Now let's calculate:

Scenario A: It's sunny, phone is charged, you have money, friends are free, but you're feeling lazy.

Inputs: [1, 1, 1, 1, 1] Calculation: (1 x 0.6) + (1 x 0.9) + (1 x 0.7) + (1 x 0.8) + (1 x -0.9) + (-0.3) = 0.6 + 0.9 + 0.7 + 0.8 - 0.9 - 0.3 = 1.8

Result: Positive number. Neuron fires. "Yes, get off the couch and go out!"

Scenario B: It's raining, phone is dead, no money, friends are busy, and you're lazy.

Inputs: [0, 0, 0, 0, 1] Calculation: (0 x 0.6) + (0 x 0.9) + (0 x 0.7) + (0 x 0.8) + (1 x -0.9) + (-0.3) = -0.9 - 0.3 = -1.2

Result: Negative number. Neuron stays quiet. "Nope. Staying home. Order food."

That's literally all a neuron does. A simple weighted sum. A yes or a no. Fire or don't fire.

Ab 86 billion baar sochiye. Har neuron apna simple decision le raha hai. Aur un sabke combination se aapka poora dimaag bana hai. Koi cheez sochna, koi feeling mehsoos karna, koi decision lena. Sab neurons ka collective dance hai.

Activation Functions - The Neuron's Personality

The activation function is what gives a neuron its personality. It decides the exact rule for "should I fire or not?"

Step Function (Hard Boundary) The original Perceptron. If the total is above zero, output is 1. Otherwise, output is 0. No grey area. Very strict.

Yeh woh teacher hai jo sirf pass ya fail batata hai. No "almost passed," no "close but not quite." Strictly binary.

Sigmoid (Smooth Curve) Output is anywhere between 0 and 1. 0.7, 0.2, 0.99. The neuron can be slightly sure, very sure, or completely uncertain.

Yeh woh dost hai jo kabhi bhi clear jawab nahi deta. "Plan ban raha hai yaar, 70 percent chance hai aane ka." Always a probability, never a yes or no.

ReLU (The Chill One) If the input is positive, pass it through as is. If it's negative, output zero. Simple, fast, effective.

Yeh woh banda hai jo positive rahta hai aur negative cheezon ko ignore karta hai. "Negativity? Don't know her."

Tanh (The Balanced One) Output is between -1 and 1. Both positive and negative signals get through, but everything is compressed into a range.

Yeh woh log hain jo extreme nahi hote. Na bohot khush, na bohot dukhi. Balance mein rehte hain.

A Quick Look Inside a Real Brain

Just to give you perspective. A single neuron in your brain fires about 200 times per second. With 86 billion neurons, that's 17 trillion firings per second. Every second. All day. All night.

Your brain consumes about 20 watts of power. That's less than a dim light bulb.

A modern AI model like GPT-4 has about 1.76 trillion parameters. It takes thousands of specialized computer chips running at hundreds of thousands of watts to run it. And it still can't tie its own shoelaces.

Biology is ridiculously efficient. We're just trying to catch up.

Nature had a 3.5 billion year head start. Give us time.

Three Fun Facts About Neurons

A single neuron can have up to 10,000 connections to other neurons. That means your brain has more connections (synapses) than there are stars in the Milky Way galaxy. About 100 trillion connections.
Neurons fire at about 200 miles per hour. That sounds fast until you realize electricity in a computer moves at close to the speed of light. Biology is slow compared to silicon. But biology makes up for it with massive parallelism.
You lose neurons as you age. By the time you're 80, you've lost about 15% of your neurons. But your artificial neurons? They never die. They don't age. They don't forget. Unless you delete the model or corrupt the file, they live forever.

The Beautiful Simplicity of It All

Here is the most mind blowing thing about all of this.

The neuron is so simple that a high school student can understand the math. Multiply some numbers. Add them up. Check if they cross a threshold. That's it.

And yet, from that simple operation, repeated billions of times with billions of connections, emerges everything we call intelligence. Consciousness. Creativity. Love. Fear. Art. Science. Everything.

The neuron is the LEGO brick of intelligence. By itself, boring. A thousand of them? You can build a castle. A billion? You can build a city. 86 billion? You can build a human mind.

Deep learning is just us copying this trick. Taking simple little math units, connecting them in clever ways, and watching intelligence emerge.

Neurons are like people. Alone, we're nothing special. But together? Together we wrote Shakespeare, built the pyramids, landed on the moon, invented pizza delivery, and created cat videos on the internet. Never underestimate what simple things can do when they work together.