Lesson 4: Neural Networks in Plain Words

A neural network is a machine-learning model loosely inspired by the brain. It is built from many small units called neurons, arranged in layers.

One neuron

A neuron takes some numbers in, multiplies each by a weight (how important that input is), adds them up, adds a bias, and then decides whether to "fire" (output 1) or not (output 0). Learning means adjusting the weights and bias.

A neuron that learns

Here is a single neuron (a perceptron) learning the AND rule: output 1 only when both inputs are 1. Each time it is wrong, it nudges its weights in the right direction.

data = [((0, 0), 0), ((0, 1), 0), ((1, 0), 0), ((1, 1), 1)]
w1, w2, bias = 0.0, 0.0, 0.0

def predict(x1, x2):
    return 1 if w1 * x1 + w2 * x2 + bias > 0 else 0

for round_number in range(10):
    for (x1, x2), target in data:
        error = target - predict(x1, x2)
        w1 += 0.1 * error * x1
        w2 += 0.1 * error * x2
        bias += 0.1 * error

for (x1, x2), target in data:
    print(x1, x2, predict(x1, x2))

Output:

0 0 0
0 1 0
1 0 0
1 1 1

It started knowing nothing and ended up answering all four cases correctly, purely by adjusting its numbers after each mistake. That is learning.

Layers and depth

• Input layer: takes the raw data (pixels of a photo, numbers, words).
• Hidden layers: many neurons that find patterns, from simple (edges) to complex (faces).
• Output layer: gives the answer (cat or dog, a number, the next word).

Deep learning just means networks with many hidden layers. A network with millions or billions of weights needs a lot of data and computing power to train, which is why training large AI models is expensive.

Think about it

1. Why does the neuron need to see the data many times (rounds)?
2. What happens to the output if all weights are 0?
3. Try the same code with the OR rule: output 1 if at least one input is 1.

Next lesson: chatbots and language models.