Lesson 2: How Machines Learn from Data

Machine learning has a simple recipe: collect examples, learn a pattern, test it, then use it on new cases.

The vocabulary

• Data: the examples. Each example has features (facts about it, like weight and size) and sometimes a label (the right answer, like "mango").
• Training: showing the learner many examples so it finds a pattern.
• Model: what is left after training: the learned pattern.
• Testing: checking the model on examples it has not seen, to see if it truly learned.

Three kinds of learning

A tiny example: guessing the fruit

We know the weight (grams) and diameter (cm) of some fruits. For a new fruit, the simplest idea is: find the closest known example and copy its label. This is called nearest neighbour. It is a real machine-learning method.

import math

# (weight in grams, diameter in cm), label
fruits = [
    ((150, 7), "apple"),
    ((170, 7.5), "apple"),
    ((120, 6), "orange"),
    ((130, 6.5), "orange"),
    ((300, 9), "mango"),
    ((320, 9.5), "mango"),
]

def guess(weight, diameter):
    best = None
    for (w, d), label in fruits:
        distance = math.sqrt((weight - w) ** 2 + (diameter - d) ** 2)
        if best is None or distance < best[0]:
            best = (distance, label)
    return best[1]

print(guess(160, 7.2))
print(guess(310, 9.2))
print(guess(125, 6.2))

Output:

apple
mango
orange

The program was never given a rule such as "mangoes weigh more than 250 g". It just compares a new fruit with examples. More (and better) examples usually mean better guesses.

Why we test on new data

A student who memorises last year's answers may fail a new paper. In the same way, a model must be tested on data it was not trained on. Otherwise we only learn that it can remember, not that it can generalise.

Think about it

1. What features would you use to tell a chicken from a duck in a photo?
2. Why would a fruit model trained only on apples fail on mangoes?
3. Add two more fruits to the example and see if the guesses change.

Next lesson: a model that predicts a number.