Loading the lesson
Loading the lesson
Search AllRounder.ai
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Learn it your way
A short class with Sarah and her students. The other ways are under More.
Sign up free to listen. Transcripts are free to read.
Here are six customers as [spending, visits]: [1, 2], [1, 4], [1, 0], [10, 2], [10, 4] and [10, 0]. What's missing compared with our pass-or-fail data?
There's no answer column. Nobody says which group each customer is in.
Exactly, that's why this is unsupervised learning. We want the computer to find the groups itself.
I can already see it: three customers spend 1 and three spend 10.
With six points, so can we. With six million customers and twenty features, we need an algorithm. Let's watch k-means do it.
We ask for k = 2 and start with two centres: customer 1 at [1, 2] and customer 6 at [10, 0]. Where does customer 4, at [10, 2], go?
It's 9 away from [1, 2] in spending but only 2 away from [10, 0] in visits, so it joins [10, 0].
Right. Do that for everyone and the first three customers join one centre and the last three join the other. Now each centre moves to the average of its group.
So they move to [1, 2] and [10, 2].
In round 2 nobody switches groups, so the centres stop moving and k-means is finished.
Just two rounds, and it found the groups without ever being told about them.
The output says labels [0, 0, 0, 1, 1, 1]. What do 0 and 1 mean?
Nothing on their own. We look at the centres: [1, 2] is the low spenders and [10, 2] is the high spenders.
Good. And both centres have 2 visits, so what really separates the groups?
Spending. Visits are the same on average.
Now suppose we had started with customers 1 and 2 as centres. k-means would split by visits instead and get stuck there, with a far worse total squared distance.
So we should try good starting points, and check the result makes sense before we use it.
The six customers have features (spending and visits) but no answers, which makes this unsupervised learning.
Assign each customer to the nearest centre, move each centre to its group's average, and repeat until nothing changes.
Two clusters with centres [1, 2] (low spenders) and [10, 2] (high spenders), found in just two rounds.
Cluster numbers are only tags, so people name the clusters by reading their centres.
A poor start can trap k-means in a worse grouping, which inertia (total squared distance) reveals.
The algorithm always returns k groups and treats every feature equally, so both choices shape the result.
Describe each cluster by its size and average values so a person can give it a name.
customers = [[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]]
labels = [0, 0, 0, 1, 1, 1]
for j in [0, 1]:
members = [c for c, label in zip(customers, labels) if label == j]
spending = sum(m[0] for m in members) / len(members)
visits = sum(m[1] for m in members) / len(members)
name = "high spenders" if spending > 5 else "low spenders"
print("Cluster", j, "|", len(members), "customers | average spending", spending, "| average visits", visits, "->", name)Place new customers by finding the nearest centre; [5, 1] is a closer call, but it still sits nearer the low spenders.
centres = {"low spenders": [1.0, 2.0], "high spenders": [10.0, 2.0]}
new_customers = [[8, 3], [2, 5], [5, 1]]
for c in new_customers:
distances = {name: (c[0] - m[0]) ** 2 + (c[1] - m[1]) ** 2 for name, m in centres.items()}
best = min(distances, key=distances.get)
print("Customer", c, "| squared distances", distances, "->", best)A bad start can trap k-means: starting from customers 1 and 2 splits by visits, and inertia shows how much worse that result is.
customers = [[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]]
def kmeans(centres, rounds=10):
for _ in range(rounds):
labels = []
for c in customers:
d = [(c[0] - m[0]) ** 2 + (c[1] - m[1]) ** 2 for m in centres]
labels.append(d.index(min(d)))
centres = []
for j in range(2):
members = [c for c, label in zip(customers, labels) if label == j]
centres.append([sum(m[0] for m in members) / len(members), sum(m[1] for m in members) / len(members)])
inertia = sum((c[0] - centres[label][0]) ** 2 + (c[1] - centres[label][1]) ** 2 for c, label in zip(customers, labels))
return labels, centres, inertia
for name, start in [("Good start", [[1, 2], [10, 0]]), ("Bad start", [[1, 2], [1, 4]])]:
labels, centres, inertia = kmeans(start)
print(name, "| labels:", labels, "| centres:", centres, "| inertia:", inertia)Pick your centres, let points choose, move to the middle, repeat till no moves.
Two ice-cream vans park on a beach. Every sunbather walks to the nearer van, then each van drives to the middle of its crowd. After a couple of moves nobody switches vans, and the beach has sorted itself into two groups.
A.U.R. - Assign, Update, Repeat.
M.E.A.N.S. - Make k centres, Everyone joins the nearest, Average each group, Next round, Stop when nothing moves.
Term
Why is clustering customers an example of unsupervised learning?
Click to flip
Term
What are the two steps k-means repeats?
Click to flip
Term
Which clusters and centres does k-means find for the six customers?
Click to flip
Term
What does the cluster number 0 or 1 tell you?
Click to flip
Term
What is inertia?
Click to flip
Learning from data that has no labels, by finding patterns and structure in the features alone.
Grouping data points so that points in the same group are more similar to each other than to points in other groups.
A clustering algorithm that repeatedly assigns points to the nearest of k centres and moves each centre to the mean of its points.
How many groups k-means is asked to find; in this example, 2.
The average point of a cluster, which stands for its typical member.
The number a clustering algorithm gives each point's group; it is a tag, not a ranking.
The total squared distance from every point to its own cluster centre; lower means tighter clusters.
Dividing customers into groups with similar behaviour so that each group can be served differently.
This worked example lets a computer sort six customers into groups using only their spending and number of visits, with no labels. K-means with k = 2 finds two clusters, low spenders and high spenders, with centres at [1, 2] and [10, 2].
Clustering is the main job of unsupervised learning: finding groups in data that has no answers attached. Here a shop describes six customers by two numbers, spending and visits, and asks k-means for k = 2 groups. The algorithm places two centres, assigns every customer to the nearest one, moves each centre to the average of its customers, and repeats until nothing changes. The result is one cluster of low spenders and one of high spenders, which both visit twice on average. We trace every round by hand, learn how to name and use the clusters, and see why a bad starting point or the wrong k can mislead the algorithm.
In the supervised examples, every student came with a correct answer. This time a shop has six customers and no labels at all: nobody has told the computer who is a big spender. The goal is to let the computer discover groups by itself. That is unsupervised learning, and grouping similar items is called clustering.
The data. Each customer is described by a pair of numbers, [spending, visits]:
Notice what is missing: there is no column saying which group anyone belongs to.
The algorithm: k-means with k = 2. We decide how many groups we want (k = 2). Each group gets a centre, a point that stands for the typical member. After choosing starting centres, k-means repeats two simple moves, assign and update:
Here it is in plain Python. Comparing squared distances picks the same nearest centre as comparing the distances themselves, and saves a square root:
# Each customer = [spending, visits]
customers = [[1, 2], [1, 4], [1, 0], [10, 2], [10, 4], [10, 0]]
k = 2
# Step 1: use the first and the last customer as starting centres
centres = [customers[0], customers[-1]]
def squared_distance(a, b):
return (a[0] - b[0]) ** 2 + (a[1] - b[1]) ** 2
for step in range(1, 11):
# Step 2: each customer joins its nearest centre
labels = []
for c in customers:
distances = [squared_distance(c, centre) for centre in centres]
labels.append(distances.index(min(distances)))
# Step 3: move each centre to the average of its customers
new_centres = []
for j in range(k):
members = [c for c, label in zip(customers, labels) if label == j]
avg_spending = sum(m[0] for m in members) / len(members)
avg_visits = sum(m[1] for m in members) / len(members)
new_centres.append([avg_spending, avg_visits])
print("Round", step, "| labels:", labels, "| centres:", new_centres)
# Step 4: stop when the centres no longer move
if new_centres == centres:
break
centres = new_centres
print("Cluster labels:", labels)
print("Centres:", centres)The result. In round 1, customers 1, 2 and 3 are closest to the centre at [1, 2], and customers 4, 5 and 6 are closest to the one at [10, 0]. For example, customer 4 at [10, 2] is 81 away (squared) from [1, 2] but only 4 away from [10, 0]. The averages then move the centres to [1.0, 2.0] and [10.0, 2.0]. In round 2 nobody changes group and the centres stay put, so the algorithm stops. The final labels are [0, 0, 0, 1, 1, 1].
What it means. The computer has found two segments:
Both groups visit equally often on average, so it is spending, not visits, that separates them. The algorithm never knew the words "low" or "high". We name the clusters by reading their centres, and the numbers 0 and 1 are just tags: another run could swap them. A shop could now send loyalty rewards to the high spenders and starter offers to the low spenders, and place any new customer in the cluster with the nearest centre.
Three cautions come with k-means:
KMeans, used in the parent lesson, guards against this by choosing well-spread starting centres (a method called k-means++), and it can run several starts and keep the one with the lowest inertia.With scikit-learn the whole example is KMeans(n_clusters=2), model.fit(data), then model.labels_ and model.cluster_centers_, which give the same two groups and centres, possibly numbered the other way round.