Loading the lesson
Loading the lesson
Search AllRounder.ai
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Search the courses, subjects, tracks, games and features, or jump straight to a page.
Learn it your way
A short class with Sarah and her students. The other ways are under More.
Sign up free to listen. Transcripts are free to read.
When your program needs data from a web API, it sends a GET request. What do you think the server does with it?
It finds the resource at that URL and sends it back?
Right. And it changes nothing on the server, which is why GET is called safe. Sending it twice has the same effect on the server as sending it once, so it is also idempotent.
So if the network drops, I can just send it again?
Exactly. A GET can be retried or cached without side effects. In Python it is one line: requests.get(url).
Suppose we only want the posts written by user 1. How could we tell the server that?
Add something to the URL, like ?userId=1?
Yes. With requests you don't build that string yourself: pass params={'userId': 1} and it encodes the query string for you.
And how do we read what comes back?
Check response.status_code first. If it is 200, call response.json() to turn the JSON body into Python lists and dictionaries.
What about response.text?
That is the raw body as a string. It is a property, so there are no parentheses.
What could go wrong when we call an API over the network?
The server might be down, or the request might hang forever.
Both happen. So we always pass a timeout, and we call raise_for_status() to turn a 404 or a 500 into an exception.
Then one try and except can catch everything?
If you catch requests.exceptions.RequestException, yes. It is the base class for timeouts, connection errors and HTTP errors.
Retrieves a resource from a server without changing server state, so it is safe and idempotent.
A dictionary that requests encodes into the URL's query string, such as ?userId=1.
Check response.status_code before using the data; 200 means OK, 4xx is a client error and 5xx a server error.
response.json() turns a JSON body into Python lists and dictionaries; response.text gives the raw string.
Pass timeout= and call raise_for_status(), then catch requests.exceptions.RequestException.
Building a GET request with only the standard library: urlencode makes the query string that params= would make, and Request holds the method, URL and headers (it is only built here, because sending it needs a network):
from urllib.parse import urlencode
from urllib.request import Request
payload = {"userId": 1, "_limit": 3}
url = "https://jsonplaceholder.typicode.com/posts?" + urlencode(payload)
req = Request(url, headers={"Accept": "application/json"}, method="GET")
print(req.get_method(), req.full_url)
print(req.header_items())A GET with query parameters against a small simulated server, following the same steps as the requests example (status check, JSON, loop over the posts):
import json
from urllib.parse import parse_qs, urlencode, urlsplit
# A tiny in-memory "server" so this runs without a network
POSTS = [
{"userId": 1, "id": 1, "title": "Reading data with GET"},
{"userId": 1, "id": 2, "title": "Query strings filter results"},
{"userId": 2, "id": 3, "title": "A post by someone else"},
]
class SimulatedResponse:
"""The parts of requests.Response that this example uses."""
def __init__(self, status_code, text):
self.status_code = status_code
self.text = text
def json(self):
return json.loads(self.text)
def simulated_get(url, params=None):
if params:
url = url + "?" + urlencode(params) # what requests does with params=
print("GET", url)
query = parse_qs(urlsplit(url).query)
user = query.get("userId", [None])[0]
found = [p for p in POSTS if user is None or str(p["userId"]) == user]
return SimulatedResponse(200, json.dumps(found))
payload = {"userId": 1}
response = simulated_get("https://jsonplaceholder.typicode.com/posts", params=payload)
if response.status_code == 200:
posts = response.json()
for post in posts:
print(post["title"])Error handling with a simulated response that has its own raise_for_status(), so a 404 becomes an exception just as it does in requests:
import json
class HTTPError(Exception):
"""Stands in for requests.exceptions.HTTPError."""
class SimulatedResponse:
def __init__(self, url, status_code, text):
self.url = url
self.status_code = status_code
self.text = text
def json(self):
return json.loads(self.text)
def raise_for_status(self):
# Like requests: status codes of 400 and above are errors
if self.status_code >= 400:
raise HTTPError("%d Client Error for url: %s" % (self.status_code, self.url))
def simulated_get(url, timeout=None):
if url.endswith("/posts/1"):
return SimulatedResponse(url, 200, '{"id": 1, "title": "Found it"}')
return SimulatedResponse(url, 404, "{}")
for url in ["https://api.example.com/posts/1", "https://api.example.com/posts/999"]:
try:
response = simulated_get(url, timeout=5)
response.raise_for_status()
print("OK:", response.json()["title"])
except HTTPError as e:
print("Error:", e)GET to read and leave things be; check the status, then parse the JSON, one, two, three.
Priya's weather app froze every time the weather server went down. She added timeout=5 and raise_for_status(), and now the app shows a friendly error message in five seconds instead of hanging forever.
"Send, Check, Parse": send requests.get(), check status_code, parse with .json().
G.E.T. - Go to the URL, Examine the status code, Take the JSON.
Term
What does a GET request do?
Click to flip
Term
Which requests argument adds query parameters to the URL?
Click to flip
Term
How do you turn a JSON response body into Python objects?
Click to flip
Term
What does `response.raise_for_status()` do?
Click to flip
Term
Why should you always pass `timeout=` to `requests.get()`?
Click to flip
An HTTP request that asks a server to return a resource without changing it.
Key and value pairs added to a URL after a question mark, such as ?userId=1, used to filter or page results.
The number a server sends back to report the result of a request, such as 200 for success or 404 for not found.
What requests.get() returns, holding the status code, headers and body of the reply.
Describes a request that has the same effect however many times it is sent.
The longest time, in seconds, a client waits for the server before giving up.
A response method that raises HTTPError when the status code is 400 or above.
A GET request is used to retrieve data from a server. Python's requests.get() lets you make such requests and read the response, which can be in text, JSON, or other formats.
The GET method is one of the most commonly used HTTP methods. It’s used to request data from a specified resource, without changing anything on the server. Query parameters can be attached using the params argument. The response usually contains the requested data, which you read with response.json() (a method that parses JSON) or response.text (a property holding the body as a string, with no parentheses).
A GET request is how a program asks a server for data. The requests library turns it into one call, requests.get(), which returns a response object holding the status code, the headers and the body. This lesson walks through that call, its parameters, query strings and error handling, using the free JSONPlaceholder test API at jsonplaceholder.typicode.com.
Because a GET changes nothing, you can repeat it, cache its result, or retry it after a network error without side effects.
Example: fetch one post and print its title. The requests code in this lesson needs the requests package (pip install requests) and network access, so it will not run in the in-browser editor. The Examples section below has standard-library versions that run anywhere.
# Needs the requests package (pip install requests) and network access
import requests
response = requests.get("https://jsonplaceholder.typicode.com/posts/1")
data = response.json()
print(data["title"])response.json() parses the JSON body into a Python dictionary, so data["title"] reads one field of the post.
The call looks like this: requests.get(url, params=None, headers=None, timeout=None)
Only url is required. Strictly, requests.get() takes url and params and passes every other keyword (such as headers and timeout) on to the request. There is no default timeout, so without one a request can wait indefinitely for a server that never answers.
Pass a dictionary as params and requests builds the query string for you: {"userId": 1} becomes ?userId=1 at the end of the URL, so the server returns only the posts written by user 1.
# Needs the requests package (pip install requests) and network access
import requests
payload = {"userId": 1}
response = requests.get("https://jsonplaceholder.typicode.com/posts", params=payload)
if response.status_code == 200:
posts = response.json()
for post in posts:
print(post["title"])Checking response.status_code == 200 before parsing means the loop only runs when the server actually sent the data back.
A real network can be slow, unreachable or answer with an error page. Set a timeout and turn bad status codes into exceptions:
# Needs the requests package (pip install requests) and network access
import requests
try:
response = requests.get("https://api.example.com/data", timeout=5)
response.raise_for_status() # Raises HTTPError for 4xx and 5xx status codes
print("Status:", response.status_code)
except requests.exceptions.RequestException as e:
print("Error:", e)raise_for_status() raises requests.exceptions.HTTPError when the status code is 400 or above (client and server errors). RequestException is the base class of every requests error, including Timeout, ConnectionError and HTTPError, so one except clause catches them all. The address api.example.com is a placeholder that does not resolve, so as written the request fails and the except branch prints the connection error; point it at a real API to see the success path.
response.status_code: the HTTP status as a number (200 means OK).response.json(): the body parsed from JSON into Python lists and dictionaries.response.text: the body as a string.response.headers: the response headers, in a case-insensitive dictionary.