Python Requests Module Tutorial

Python Requests

This is a detailed tutorial of the Python Requests Module. Learn to send any kind of requests (GET, PUT, POST, DELETE, etc.) and to get their response.

Python Requests

In most of the applications these days, we need to get the data from other sources by sending them different requests. The requests are also used for performing different actions of the servers and systems. In python, we have got a module named requests that can be used to send any kind of requests like GET, PUT, POST, DELETE, PATCH, HEAD or any other custom request. The Python Requests module offers a complete solution to handle the request-response as well. In other words, this module is a perfect solution to send any kind of HTTP request.

I have my hands-on different programming languages and I know in some of the programming languages, you need to do a lot of struggle to send the requests and to process their response. But in python, the Python Requests library makes things simpler and better than ever. This module offers simple methods to send different kinds of requests and you can easily pass the request data using arguments and the response is provided as the returned data of these methods.

Syntax

The basic syntax for all the different methods that you can use under this library is given below.

Note. The keyword method in the above syntax has to be replaced with the actual method name that we’ll discuss further in this article. The response so returned will be stored into the variable named response and you can apply a number of different methods to fetch the response information appropriately.

The first parameter is always the request URL and it is also a mandatory argument. All other parameters are optional. In the second parameter, you can send data in the form of key-value pairs. You can also provide a Python Dictionary here as the second argument.

The third argument **kwargs is actually used to represent zero or more arguments. There are many more different arguments that can be specified within the different methods of the Python Requests Module. These additional arguments are can be used to specify additional data, files, timeout, cookies, proxies, streams, etc. We’ll find more about the usage of these things in a later section of this article. It returns a response object. We can apply a number of properties and methods to this response object to get valuable data.

You can find out more about the *args and **kwargs notation at the following linked article.

Request Methods

In this article, we’ll talk discuss seven different methods that you can use with the Python Requests library for the purpose of sending different requests and to process their responses.

GET

Mostly, we need to send GET requests to fetch data from some resource or URL. The python requests library offers get() a method to send a GET request and to receive its response.

Example. The following python program sends a GET request to a URL to prints out the response code and the textual data so received.

The RatesAPI is a FREE API to fetch Foreign Exchange Rates and here we’re sending a GET request to the API URL using the Python Requests Library and then printing out the response status code and response text. Apart from the response code and text, there are a lot of different methods that can be applied to the response object to get more valuable information. We’ll also discuss these response object methods at the end of this article.

Python Requests GET Example

POST

The HTTP POST method is usually used to send requests that are intended to store or insert the data into the database. In other words, the POST requests are used for submission purposes. In these requests, we send along with some data in different forms that are meant to be submitted or stored.

Example. The following python code sends a POST request to a URL. This URL is actually a dummy URL to test POST requests. Therefore, it also returns the arguments and data that we sent along with the POST request using the Python Requests Module post method.

We’ve stored the data to send in a dictionary stored in the variable info and now we’re sending this variable as the second argument to the post method. Observe carefully the request-response text shown in the following screenshot, the data that we sent is also returned.

Python Requests POST Example

PUT

The PUT method is usually used to send the information that is meant to be updated in the database. In the Python Requests module, the method put() can be used to send such requests.

Example. The following python code sends a PUT request to a dummy URL along with arguments and body data.

Python Requests PUT Method Example

DELETE

The delete method is used to send the server a request to delete some record(s) from the database. The method delete() can be used with Python Requests module to simulate an HTTP DELETE Request.

Example. The following code snippet sends a DELETE request to a dummy Test URL.

In this DELETE request, we’ve sent the id of the database record to be deleted as the request data.

Python Requests DELETE Method Example

HEAD

The method head() is used with the Python Requests module to simulate an HTTP HEAD request. Unlike the other methods like GET, POST, PUT and DELETE, this method does not take the second argument data. But it accepts the other **kwargs. The HEAD method is used to get the request-response status or code and not the content, therefore the data parameter is not required.

Example. The following code sends a head request to a URL just to check its status code.

We’ve also written the line of code to print the textual data, but as it is a HEAD request, no textual data is returned.

Python Requests HEAD Method Example

PATCH

A PATCH request is also used to patch or update the server resources or data. The method patch() simulates PATCH requests in Python Requests library.

Example. A PATCH request is being sent with some data in the following code snippet.

Python Requests PATCH Method Example

CUSTOM REQUEST

In case you want to send a request with a CUSTOM method, you can use the request() method to send any kind of request. The first parameter of the method request() is a string defining the name of the method itself. This way you get the flexibility of sending requests to your own created methods.

Example. In the following code, we’re simulating the GET Method request using the method request() instead of using the get() method.

Request Method Parameters

As denoted by the third syntax argument **kwargs that any of the above methods can take a number of variable parameters. This section describes those parameters and their purpose with the help of an example for each.

Parameter Description Example
url The only required parameter defines the URL to which the request has to be sent. requests.get(“https://postman-echo.com/get”)
params Optional.

You can provide key-value pair data here in the form of a Python Dictionary. List of Tuples, Query String bytes can also be provided.

Default value is None.

info = {“id”: 5, “name”: “Gurmeet Singh”, “age”: 21}

requests.post(“https://postman-echo.com/post”, info)

allow_redirects Optional Boolean parameter.

Specifies whether you want the request URL to be redirected to another Request URL or not.

Default value is True.

requests.head(“https://postman-echo.com/get”, allow_redirects = False)
auth Optional Tuple parameter.

Used for HTTP authentication.

Default value is None.

requests.get(url, auth = (‘username’, ‘password’))

or

requests.get(url, auth = (‘<api-key>’))

cert Optional String or Tuple Parameter.

Specified a certificate or a key for the current request.

Default value is None.

requests.put(“https://postman-echo.com/get”, cert = “some-location/certificate-file.cert”)
cookies Optional Dictionary Parameter.

Used to specify cookies to send along with the current request.

Default value is None.

some_cookies = {“id”: 5, “name”: “Gurmeet Singh”, “age”: 21}

requests.post(“https://postman-echo.com/post”, cookies = some_cookies)

headers Optional Dictionary Parameter.

Used to specify the different headers to be sent for the current request.

Default value is None.

my_headers = {“Content-Type”: “application/json; charset=utf-8”}

requests.delete(“https://postman-echo.com/post”, headers = my_headers)

proxies Optional Dictionary Parameter.

Used to specify the proxies in the key-value form for the current request.

Default value is None.

requests.get(url, proxies = { “https” : “<some-IP-address”})
stream Optional Boolean Parameter.

Used to specify whether the response should be immediately downloaded into the response object or it has to be streamed.

Useful for API requests that respond with streaming videos.

False means Download and True meaning stream.

Default value is False.

requests.get(“http://some-video-api.com/video?id=546”, stream=True)
timeout Optional Numerical or Tuple Parameter.

Used to specify the time a request should wait to get its response.

The time has to be specified in seconds. 0.05 represents 50 milliseconds.

Default value is None indicating the request has no defined response wait time and it will keep on continuing until the response is received.

requests.head(“https://postman-echo.com/get”, timeout = 0.05)
verify Optional Boolean Parameter.

Used to verify a Certificate. Specifying it as False will not verify the certificate.

Default value is True.

requests.get(“https://postman-echo.com/get”, verify=’some-location/some-certificate’)

Response Object Properties and Methods

We know that all of the methods return a Response Object. The response object itself is not giving any information, but you need to make use of some properties and methods to get the right and required information contained in this response object. These different methods and properties are given below in the tabular form. Instead of giving example for each of the response object properties and methods, I’ve given a full example code afterward that makes use of each of these.

Property or Method Description
apparent_encoding Property. Returns the apparent encoding for the response.
close() Method. Used to close the current connection. Quite useful for streaming responses.
content Property. Returns the content in bytes form.
cookies Property. It returns the cookie Jar object that contains the cookies sent back by the server.
elapsed Property. It returns a timedelta object that contains the time to process the request, i.e. the time interval between the sending instance and response received instance.
encoding Property. It returns the encoding that is used to decode the textual data so returned.
headers Property. The response headers are being stored in a Python Dictionary and is being returned by this property.
history Property. This returns the history of request URL(s). In case the request is hoped from one URL to another, the history property can be used to know the redirected URLs.
is_permanent_redirect Property. It returns a Boolean value; True if the URL is permanently redirected and False if not.
is_redirect Property. It will return True if the request URL redirects at least once, otherwise False.
iter_content() Method. It can be used to loop through the content of the response.
iter_lines() Method. The response can be iterated line by line using this method.
json() Method. If and only if the response is in JSON form, it returns the response as a JSON object, otherwise, it raises an error.
links Property. It returns the links of the header.
next Property. If the request is meant for a redirection it returns an object that is already prepared to be sent in the next redirected request.
ok Property. If the status code is 200 or less than 200, it returns true, otherwise False.
raise_for_status() Property. This returns an HTTPError object if an error occurs.
reason Property. Returns the message associated with the response code.
request Property. The request object is returned that is being used to get the response for the current request.
status_code Property. It returns the response status code. For example, 200, 201, 404, etc.
text Property. It returns the textual response in Unicode form.
url Property. This returns the URL for the response.

Example. The following python program illustrates the use of all of the above-defined Python Request Module’s Response Object’s Properties and methods for a simple GET request.

As tried to print the response of all the properties and methods, the output is very long. Only some part of the output is shown in the following screenshot.

Python Requests Method Response Object Properties And Methods Example

Related Guides.

I hope you found this guide useful. If so, do share it with others who are willing to learn Python. If you have any questions related to this article, feel free to ask us in the comments section.

And do not forget to subscribe to WTMatter!

Subscribe To WTMatter!

Receive updates of our latest articles via email. Enter your email address below to get started.

Leave a Reply

Your email address will not be published. Required fields are marked *