Loading the lesson
Loading the lesson
Search AllRounder.ai
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Learn it your way
A short class with Sarah and her students. The other ways are under More.
Sign up free to listen. Transcripts are free to read.
Last time we predicted marks. This time the question is simply: will a student pass or fail?
So the answer is a word, not a number?
Exactly, a category. We code Fail as 0 and Pass as 1, but those are just names. That makes it classification.
Our data says 1 and 2 hours failed, and 3, 4 and 5 hours passed.
Good. Now a new student studied 2.5 hours. How could we decide without a formula?
Look at the students who studied about the same and go with what happened to them?
That is exactly the idea behind K-Nearest Neighbours.
With k = 3 we find the three closest students. How far is 2.5 hours from each one?
2 and 3 hours are 0.5 away, 1 and 4 hours are 1.5 away, and 5 hours is 2.5 away.
So the two nearest are 2 hours, who failed, and 3 hours, who passed. Who gets the third seat?
It's a tie! 1 hour and 4 hours are both 1.5 away.
Right. Our code keeps the one that comes first in the list, the 1-hour student, so the vote is Fail 2, Pass 1.
So the prediction is Fail, but only because of how the tie was broken.
Why is 2.5 hours such a hard case?
It's exactly halfway between the last Fail at 2 hours and the first Pass at 3 hours.
Yes, it sits on the decision boundary. Compare 4.2 hours: all three neighbours passed, so that answer is solid.
What if we used all five students, k = 5?
Then every student votes every time, Pass wins 3 to 2, and the model says Pass even for 1 hour. Too large a k ignores the local pattern.
So we pick a small odd k, and treat close votes as uncertain.
Classification predicts which class a new example belongs to, here Pass or Fail.
KNN stores the training examples and does its work only when a prediction is needed.
Measure distances, sort them, keep the k nearest and take a majority vote.
2.5 hours lies on the decision boundary, so its prediction depends on how a distance tie is broken.
A 3-to-0 vote is a firm prediction, while a 2-to-1 vote is only a lean.
Use a small odd k for two classes; an odd k avoids tied votes, and a very large k just predicts the most common class.
Predicting several new students with k = 3 shows clear cases on both sides of the boundary and the fragile one in the middle.
hours = [1, 2, 3, 4, 5]
results = [0, 0, 1, 1, 1]
def knn_predict(new_hours, k=3):
ranked = sorted(zip(hours, results), key=lambda pair: abs(new_hours - pair[0]))
votes = [r for _, r in ranked[:k]]
label = "Pass" if sum(votes) > k / 2 else "Fail"
return label, votes
for new in [1.5, 2.5, 3.4, 4.2]:
label, votes = knn_predict(new)
print(new, "hours -> neighbour results", votes, "->", label)Listing the same five students in reverse order changes who wins the tied third seat, so the 2.5-hour prediction flips while a clear case such as 4.2 hours does not.
def knn_predict(new_hours, hours, results, k=3):
ranked = sorted(zip(hours, results), key=lambda pair: abs(new_hours - pair[0]))
votes = [r for _, r in ranked[:k]]
return "Pass" if sum(votes) > k / 2 else "Fail"
hours = [1, 2, 3, 4, 5]
results = [0, 0, 1, 1, 1]
print("2.5 hours, data in the original order:", knn_predict(2.5, hours, results))
print("2.5 hours, same data listed in reverse:", knn_predict(2.5, hours[::-1], results[::-1]))
print("4.2 hours, original order:", knn_predict(4.2, hours, results))
print("4.2 hours, reversed order:", knn_predict(4.2, hours[::-1], results[::-1]))A different classifier on the same data: a threshold placed halfway between the last Fail and the first Pass lands exactly on 2.5 hours, confirming that 2.5 is the boundary.
hours = [1, 2, 3, 4, 5]
results = [0, 0, 1, 1, 1]
last_fail = max(h for h, r in zip(hours, results) if r == 0)
first_pass = min(h for h, r in zip(hours, results) if r == 1)
threshold = (last_fail + first_pass) / 2
print("Learned threshold:", threshold, "hours")
for new in [1.5, 2.4, 2.6, 4.2]:
print(new, "hours ->", "Pass" if new > threshold else "Fail")Measure, sort and count the three; the biggest group is the label you'll see.
A pupil who studied 2.5 hours asked the classmates nearest to her in study time how their exam went. The 2-hour classmate had failed and the 3-hour classmate had passed. Two more, at 1 and 4 hours, were equally close, and only the one who answered first, a fail, got the deciding vote. She realised she sat right on the line between the two groups.
M.S.K.V. - Measure, Sort, Keep k, Vote.
K.N.N. - Keep the Nearest Neighbours, then let them vote.
Term
What kind of output does a classification model give?
Click to flip
Term
What are the four steps KNN takes to make a prediction?
Click to flip
Term
Which students are the 3 nearest neighbours of a 2.5-hour student?
Click to flip
Term
Why is the 2.5-hour prediction fragile?
Click to flip
Term
What goes wrong with k = 5 on this data?
Click to flip
A supervised learning task where the output is a category, such as Pass or Fail.
The category attached to a training example; here 0 means Fail and 1 means Pass.
A classifier that predicts by letting the k most similar training examples vote.
How many nearest examples take part in the vote; this example uses 3.
How far apart two examples are; with one feature it is the difference between the values, ignoring the sign.
Choosing the class that appears most often among the nearest neighbours.
The point where predictions switch from one class to the other; here, halfway between 2 and 3 hours.
This worked example teaches a computer to predict a category, Pass or Fail, from the hours a student studied. It uses K-Nearest Neighbours with k = 3: the three most similar students vote, and the majority decides.
Classification is the part of supervised learning that predicts a category. Here five students' study hours are the input and their result, Fail (0) or Pass (1), is the label. K-Nearest Neighbours (KNN) predicts for a new student by finding the 3 students with the closest study hours and taking a majority vote. We walk through the 2.5-hour student from the parent lesson step by step and discover that 2.5 hours is a borderline case: it sits exactly between the last Fail and the first Pass, so the deciding third vote comes down to a tie. We also see clear cases, how the choice of k changes the answer, and what the prediction does and does not tell us.
In Example 1 the model predicted a number: marks. Now we ask a different question about the same kind of student: will they pass or fail? The answer is a category, not a number, so this is a classification task.
The data. Five students, each with the correct result attached:
The 0 and 1 are only names for the two classes. Unlike marks, 1 is not "more" than 0 in any useful sense; it simply means Pass.
The algorithm: K-Nearest Neighbours (KNN) with k = 3. KNN does almost nothing during training: it just remembers the labeled examples. All the work happens when a new student arrives:
Here are those four steps in plain Python for the parent lesson's question, a student who studied 2.5 hours:
# Hours studied and results for five students: 0 = Fail, 1 = Pass
hours = [1, 2, 3, 4, 5]
results = [0, 0, 1, 1, 1]
names = {0: "Fail", 1: "Pass"}
def knn_predict(new_hours, k=3):
# Step 1: distance from the new student to every known student
rows = []
for h, r in zip(hours, results):
rows.append((abs(new_hours - h), h, names[r]))
# Step 2: sort by distance (Python keeps tied rows in their original order)
rows.sort(key=lambda row: row[0])
# Step 3: the k nearest students vote
nearest = rows[:k]
for distance, h, result in nearest:
print(" neighbour who studied", h, "h | distance", distance, "->", result)
passes = sum(1 for row in nearest if row[2] == "Pass")
fails = k - passes
print(" votes: Pass", passes, "| Fail", fails)
return "Pass" if passes > fails else "Fail"
print("Prediction for 2.5 hours:", knn_predict(2.5))The result. The distances from 2.5 hours are 1.5 (1 hour), 0.5 (2 hours), 0.5 (3 hours), 1.5 (4 hours) and 2.5 (5 hours). The two nearest students split the vote: 2 hours failed and 3 hours passed. The third seat is a tie at distance 1.5 between the 1-hour student (Fail) and the 4-hour student (Pass). Python's sort keeps tied students in their original order, so the 1-hour student takes the seat. The vote is Fail 2, Pass 1, and the prediction is Fail.
What it means. That Fail is a fragile answer. 2.5 hours lies exactly halfway between the last student who failed (2 hours) and the first who passed (3 hours), which is where this data's decision boundary sits. If the 4-hour student had taken the third seat, the answer would have been Pass. The honest reading is "this data cannot tell". Students further from the boundary get clearer answers: 4.2 hours gets all three Pass votes, and 1.5 hours is voted Fail by 2 to 1 without any tie. The share of votes works as a rough confidence: 3 to 0 is sure, 2 to 1 is only a lean.
A few lessons come from this tiny example:
In real projects, scikit-learn's KNeighborsClassifier(n_neighbors=3) from the parent lesson runs the same four steps when you call predict. Libraries have their own rules for breaking ties, so do not read much into any single borderline prediction.