Python Requests Module

The requests module in Python is a popular and easy-to-use library for making HTTP requests. It abstracts away many complexities of handling HTTP, providing a simple interface for sending HTTP requests, handling responses, and interacting with APIs. The module supports various HTTP methods, including GET, POST, PUT, DELETE, and more.

This guide will introduce you to the basics of using the requests module for interacting with web resources.

1. Installing the requests Module

If you don’t have the requests module installed, you can install it using pip:

pip install requests

2. Making a GET Request

The GET method is used to retrieve data from a server. It’s the most common type of HTTP request.

Example: Sending a GET Request

import requests

response = requests.get("https://jsonplaceholder.typicode.com/posts")
print(response.status_code)  # Outputs: 200 (success)
print(response.text)  # Outputs the response body as a string

• status_code: Returns the HTTP status code (e.g., 200 for success).
• text: Returns the content of the response as a string.
• json(): If the response is JSON, this method parses and returns the data as a dictionary.

Example: Accessing JSON Response

response = requests.get("https://jsonplaceholder.typicode.com/posts")
json_data = response.json()  # Converts JSON response to a dictionary
print(json_data)

3. Sending a POST Request

The POST method is used to send data to a server, typically used for submitting form data or creating resources.

Example: Sending a POST Request

import requests

data = {
    "title": "foo",
    "body": "bar",
    "userId": 1
}

response = requests.post("https://jsonplaceholder.typicode.com/posts", json=data)
print(response.status_code)  # Outputs: 201 (created)
print(response.json())  # Outputs the server response

• json=: Sends data as JSON in the body of the request.
• data=: Sends form data (key-value pairs).

4. Sending a PUT Request

The PUT method is used to update an existing resource.

Example: Sending a PUT Request

import requests

data = {
    "id": 1,
    "title": "foo",
    "body": "bar",
    "userId": 1
}

response = requests.put("https://jsonplaceholder.typicode.com/posts/1", json=data)
print(response.status_code)  # Outputs: 200 (OK)
print(response.json())  # Outputs the updated resource

5. Sending a DELETE Request

The DELETE method is used to remove a resource from the server.

Example: Sending a DELETE Request

import requests

response = requests.delete("https://jsonplaceholder.typicode.com/posts/1")
print(response.status_code)  # Outputs: 200 (OK) or 204 (No Content)

6. Handling Query Parameters

Query parameters are added to the URL to pass additional data. You can pass query parameters using the params argument in requests.

Example: Sending Query Parameters

import requests

params = {
    "userId": 1
}

response = requests.get("https://jsonplaceholder.typicode.com/posts", params=params)
print(response.status_code)  # Outputs: 200 (success)
print(response.json())  # Outputs the filtered posts with userId=1

7. Handling Headers

You can customize HTTP headers using the headers argument to send additional information to the server, such as authorization tokens or content types.

Example: Sending Custom Headers

import requests

headers = {
    "User-Agent": "my-app",
    "Authorization": "Bearer <token>"
}

response = requests.get("https://jsonplaceholder.typicode.com/posts", headers=headers)
print(response.status_code)  # Outputs: 200 (success)

8. Handling Timeouts

When making a request, you can specify a timeout to prevent the program from hanging indefinitely if the server doesn’t respond.

Example: Using a Timeout

import requests

try:
    response = requests.get("https://jsonplaceholder.typicode.com/posts", timeout=5)
    print(response.status_code)
except requests.exceptions.Timeout:
    print("The request timed out.")

9. Handling Errors

The requests module raises exceptions for different types of errors. You can handle these exceptions using try-except blocks.

Common Exceptions:

• requests.exceptions.RequestException: A base class for all exceptions raised by the requests module.
• requests.exceptions.HTTPError: Raised when a bad HTTP response (e.g., 4xx or 5xx) is returned.
• requests.exceptions.Timeout: Raised when the request times out.

Example: Handling HTTP Errors

import requests

response = requests.get("https://jsonplaceholder.typicode.com/nonexistent")

try:
    response.raise_for_status()  # Raises HTTPError for bad responses (4xx or 5xx)
except requests.exceptions.HTTPError as err:
    print(f"HTTP error occurred: {err}")
except requests.exceptions.RequestException as err:
    print(f"Error occurred: {err}")

10. Handling Authentication

The requests module provides several ways to authenticate requests, such as basic authentication or using OAuth tokens.

Example: Basic Authentication

from requests.auth import HTTPBasicAuth

response = requests.get("https://httpbin.org/basic-auth/user/pass", auth=HTTPBasicAuth('user', 'pass'))
print(response.status_code)  # Outputs: 200 (OK)

Example: Bearer Token Authentication

headers = {
    "Authorization": "Bearer <your-token>"
}

response = requests.get("https://api.example.com/protected-resource", headers=headers)
print(response.status_code)

11. Multipart File Uploads

You can upload files to a server using the files argument in the requests module.

Example: Uploading a File

files = {'file': open('example.txt', 'rb')}
response = requests.post("https://httpbin.org/post", files=files)
print(response.status_code)
files['file'].close()

Conclusion

The requests module makes interacting with HTTP resources in Python easy and intuitive. Whether you’re fetching data, sending data, or performing advanced tasks like authentication and file uploads, requests provides a simple API for all your HTTP needs. By mastering the methods and features provided by the requests module, you can efficiently work with REST APIs, web scraping, and any other task that requires making HTTP requests.