The Python Oracle

Timeout for python requests.get entire response

--------------------------------------------------
Rise to the top 3% as a developer or hire one of them at Toptal: https://topt.al/25cXVn
--------------------------------------------------

Music by Eric Matyas
https://www.soundimage.org
Track title: Lost Civilization

--

Chapters
00:00 Timeout For Python Requests.Get Entire Response
00:38 Answer 1 Score 606
01:23 Accepted Answer Score 168
01:41 Answer 3 Score 152
03:33 Answer 4 Score 79
04:19 Thank you

--

Full question
https://stackoverflow.com/questions/2196...

--

Content licensed under CC BY-SA
https://meta.stackexchange.com/help/lice...

--

Tags
#python #timeout #pythonrequests

#avk47



ANSWER 1

Score 606


Note: The timeout param does NOT prevent the request from loading forever, it only stops if the remote server fails to send response data within the timeout value. It could still load indefinitely.

Set the timeout parameter:

try:
    r = requests.get("MYURL.com", timeout=10) # 10 seconds
except requests.exceptions.Timeout:
    print("Timed out")

The code above will cause the call to requests.get() to timeout if the connection or delays between reads takes more than ten seconds.

The timeout parameter accepts the number of seconds to wait as a float, as well as a (connect timeout, read timeout) tuple.

See requests.request documentation as well as the timeout section of the "Advanced Usage" section of the documentation.




ACCEPTED ANSWER

Score 168


What about using eventlet? If you want to timeout the request after 10 seconds, even if data is being received, this snippet will work for you:

import requests
import eventlet
eventlet.monkey_patch()

with eventlet.Timeout(10):
    requests.get("http://ipv4.download.thinkbroadband.com/1GB.zip", verify=False)



ANSWER 3

Score 152


UPDATE: https://requests.readthedocs.io/en/master/user/advanced/#timeouts

In new version of requests:

If you specify a single value for the timeout, like this:

r = requests.get('https://github.com', timeout=5)

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

r = requests.get('https://github.com', timeout=(3.05, 27))

If the remote server is very slow, you can tell Requests to wait forever for a response, by passing None as a timeout value and then retrieving a cup of coffee.

r = requests.get('https://github.com', timeout=None)

My old (probably outdated) answer (which was posted long time ago):

There are other ways to overcome this problem:

1. Use the TimeoutSauce internal class

From: https://github.com/kennethreitz/requests/issues/1928#issuecomment-35811896

import requests from requests.adapters import TimeoutSauce

class MyTimeout(TimeoutSauce):
    def __init__(self, *args, **kwargs):
        connect = kwargs.get('connect', 5)
        read = kwargs.get('read', connect)
        super(MyTimeout, self).__init__(connect=connect, read=read)

requests.adapters.TimeoutSauce = MyTimeout

This code should cause us to set the read timeout as equal to the connect timeout, which is the timeout value you pass on your Session.get() call. (Note that I haven't actually tested this code, so it may need some quick debugging, I just wrote it straight into the GitHub window.)

2. Use a fork of requests from kevinburke: https://github.com/kevinburke/requests/tree/connect-timeout

From its documentation: https://github.com/kevinburke/requests/blob/connect-timeout/docs/user/advanced.rst

If you specify a single value for the timeout, like this:

r = requests.get('https://github.com', timeout=5)

The timeout value will be applied to both the connect and the read timeouts. Specify a tuple if you would like to set the values separately:

r = requests.get('https://github.com', timeout=(3.05, 27))

kevinburke has requested it to be merged into the main requests project, but it hasn't been accepted yet.




ANSWER 4

Score 79


Since requests >= 2.4.0, you can use the timeout argument, i.e:

requests.get('https://duckduckgo.com/', timeout=10)

You can also provide a tuple to specify connect and the read timeouts separately:

requests.get('https://duckduckgo.com/', timeout=(5, 8.5))

a None timeout will wait forever (not recommended)


Note:

timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds ( more precisely, if no bytes have been received on the underlying socket for timeout seconds). If no timeout is specified explicitly, requests do not time out.