Loading the lesson
Loading the lesson
Search AllRounder.ai
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Learn it your way
A short class with Sarah and her students. The other ways are under More.
Sign up free to listen. Transcripts are free to read.
Here are five students: 1 hour of study gave 35 marks, 2 hours gave 45, then 55, 65 and 75. What do you notice?
Every extra hour adds exactly 10 marks.
Right. If we walk the pattern back to 0 hours, where do we land?
35 minus 10, so 25 marks.
So the whole rule is marks = 10 × hours + 25. The 10 is the slope and the 25 is the intercept.
Then for 6 hours it's 10 × 6 + 25, which is 85.
Exactly. A regression model finds this rule for us, even when the pattern is not this tidy.
With messy real data, no line passes through every point. How should the computer choose one?
Pick the line that is closest to all the points overall?
Yes. For each student it takes the error, real marks minus predicted marks, squares it and adds them all up.
Why square them instead of just adding the errors?
A miss above the line and a miss below it would cancel out. Squaring makes every error positive and punishes big misses more. The line with the smallest total is the best-fit line; this method is called least squares.
And for our five students the total is zero, because the line hits every point.
Our model predicts 85 marks for 6 hours. What does it predict for 10 hours?
10 × 10 + 25, so 125 marks.
On a paper out of 100. What went wrong?
The line only learned from 1 to 5 hours. It doesn't know that marks stop at 100.
Exactly. Predicting far outside the training data is called extrapolation. Also remember the line shows a pattern, not a promise: hours alone don't decide marks.
So we trust it most near the hours it has already seen.
The model predicts a number on a continuous scale, here a student's marks.
Linear regression predicts with marks = slope × hours + intercept.
The best-fit line is the one with the smallest total of squared errors.
For this data the slope is 10 marks per hour and the intercept is 25 marks.
Plugging 6 hours into the line gives 10 × 6 + 25 = 85 marks.
Far outside the training range of 1 to 5 hours, the line can give impossible answers.
Why this line is the "best" fit: compare the total squared error of three candidate lines.
hours = [1, 2, 3, 4, 5]
marks = [35, 45, 55, 65, 75]
def total_squared_error(slope, intercept):
total = 0
for x, y in zip(hours, marks):
error = y - (slope * x + intercept)
total += error ** 2
return total
candidates = [(8, 30), (12, 20), (10, 25)]
for slope, intercept in candidates:
print("marks =", slope, "x hours +", intercept, "-> total squared error:", total_squared_error(slope, intercept))Real data is noisy: with marks of 38, 41, 58, 62 and 76, least squares still finds a line close to 10 marks per hour, and every point now has a small error.
hours = [1, 2, 3, 4, 5]
marks = [38, 41, 58, 62, 76]
n = len(hours)
mean_x = sum(hours) / n
mean_y = sum(marks) / n
slope = sum((x - mean_x) * (y - mean_y) for x, y in zip(hours, marks)) / sum((x - mean_x) ** 2 for x in hours)
intercept = mean_y - slope * mean_x
print("Slope:", round(slope, 2), "| intercept:", round(intercept, 2))
for x, y in zip(hours, marks):
guess = slope * x + intercept
print("Hours:", x, "| real:", y, "| line says:", round(guess, 1), "| error:", round(y - guess, 1))
print("Prediction for 6 hours:", round(slope * 6 + intercept, 1))A straight line does not know that the paper is out of 100, so predictions far beyond the 5 hours in the data break down.
slope = 10
intercept = 25
for h in [6, 7, 8, 10]:
predicted = slope * h + intercept
warning = " <- impossible, the paper is out of 100" if predicted > 100 else ""
print(h, "hours ->", predicted, "marks" + warning)Square every miss and add the lot; the smallest sum is the line you've got.
Maya kept a notebook of revision hours and test marks. Each extra hour added about ten marks, so she drew a ruler line through her dots and read off 85 for a six-hour week. When her brother asked about ten hours, the line said 125, and they laughed: the paper only goes to 100.
S-I-P: Slope, Intercept, Predict - find the two numbers, then plug in the new input.
L.I.N.E. - Look at the data, Identify slope and intercept, Note the errors, Estimate new values.
Term
What does the model predict for 6 hours of study?
Click to flip
Term
What does the slope of 10 mean in this example?
Click to flip
Term
What does the least-squares method make as small as possible?
Click to flip
Term
What is a residual?
Click to flip
Term
Why is a prediction of 125 marks for 10 hours not trustworthy?
Click to flip
A value the model uses to make a prediction; here, the hours a student studied.
The correct output attached to each training example; here, the marks scored.
The straight line that sits closest to all the training points, measured by the total of squared errors.
How much the predicted output changes when the input goes up by one; here, 10 marks per hour.
The value of the line when the input is zero; here, 25 marks.
The method that chooses the line with the smallest sum of squared errors.
The real value minus the predicted value for one example.
Predicting for inputs outside the range of the training data, which can give unrealistic results.
This worked example teaches a computer to predict a student's marks from the hours they studied. It finds the best-fit straight line through five labeled examples, marks = 10 × hours + 25, and uses it to predict 85 marks for 6 hours.
Regression is the part of supervised learning that predicts a number. Here five students' study hours (the input) and marks (the output) are the labeled data. The model looks for the straight line marks = slope × hours + intercept that sits closest to all the points, using a method called least squares. For this data the line is marks = 10 × hours + 25, so the prediction for 6 hours is 85 marks. We also check how well the line fits, see how it copes with noisy data, and learn why predictions far outside the training data can be nonsense.
In the parent lesson you met supervised learning and its two subtypes. This lesson walks through the first worked example from start to finish: predicting a student's marks (a number) from the hours they studied. Because the answer is a number on a continuous scale, this is a regression task.
The data. Five students, each with a correct answer attached (labeled data):
Hours studied is the feature (input). Marks is the label (the output we want to predict).
Step 1: Spot the pattern. Each extra hour adds exactly 10 marks. Following the pattern back to 0 hours gives 25 marks. So the rule hiding in the table is marks = 10 × hours + 25.
Step 2: Choose a model. Linear regression assumes the output changes along a straight line: marks = slope × hours + intercept. The slope says how many marks one extra hour adds. The intercept is the line's value at 0 hours.
Step 3: Fit the line with least squares. For any candidate line, the computer measures each student's error (real marks minus predicted marks), squares it and adds the squares up. The best-fit line is the one with the smallest total. For a single input there is a short recipe:
Here is that recipe in plain Python, with no libraries:
# Five students: hours studied (input) and marks scored (output)
hours = [1, 2, 3, 4, 5]
marks = [35, 45, 55, 65, 75]
n = len(hours)
mean_x = sum(hours) / n
mean_y = sum(marks) / n
print("Average hours:", mean_x, "| average marks:", mean_y)
# Least squares: slope = sum((x - mean_x) * (y - mean_y)) / sum((x - mean_x) ** 2)
top = 0
bottom = 0
for x, y in zip(hours, marks):
top += (x - mean_x) * (y - mean_y)
bottom += (x - mean_x) ** 2
slope = top / bottom
intercept = mean_y - slope * mean_x
print("Slope (extra marks per hour):", slope)
print("Intercept (marks at 0 hours):", intercept)
print("Prediction for 6 hours:", slope * 6 + intercept)The averages are 3 hours and 55 marks. The top of the slope fraction is 100 and the bottom is 10, so the slope is 10.0 and the intercept is 55 − 10 × 3 = 25.0.
Step 4: Predict. For a new student who studies 6 hours: 10 × 6 + 25 = 85 marks. The model has "seen how marks increase with hours" and continued the same pattern one step further.
What the result means. Each extra hour of study goes with about 10 more marks, and the line passes exactly through all five points, so every training error is 0. Real data is never this tidy: marks wobble because of sleep, the difficulty of the paper and luck. Least squares then draws the line that is closest overall, and some errors remain. Two warnings come with any regression line:
In real projects you rarely type the formula yourself. In the parent lesson, scikit-learn's LinearRegression does the same job: model.fit(X, y) finds the slope and intercept, and model.predict([[6]]) gives about 85. Knowing what happens inside fit helps you read and question its output.