Concurrent requests in Appengine Python

You still only have one thread per request - you can't spawn.

With threadsafe off, Appengine will only route one request to an instance. So if the number of requests per second times the time to handle a request approaches one, Appengine will spin up a new instance to handle them. This cost money. With threadsafe on, Appengine can route more than one request to an instance.

Whether this helps you or not depends on your app and your traffic:

1. First, calculate inbound request per second / average latency. If this is well under one, threadsafe won't make much difference either way.
2. Examine your app to find out how much time it spends waiting on APIs (datastore or URL fetch for instance). If this is a large proportion, then threadsafe will help keep your instance count down. If not, it won't help much.

The simple rule is switch threadsafe on unless your app is very processing-intensive (little API waiting).

1. It doesn't mean that your application will become faster, the request are still served from a single thread.
2. When the application is thread safe each instance can now spawns multiple threads each thread will serve a request as apposed to the non thread safe where each instance has a single thread serving requests.

Python 2.5 is still a bit faster, on a per-request basis, than Python 2.7. That's partially due to how mature each is. App Engine uses different mechanisms to support each of them. The win with Python 2.7's is its ability to support parallel requests rather than spinning up new instances at a rate that would be required by Python 2.5 to handle load spikes.

The "how does it work internally" question is one that you're probably not going to get an answer for here, but there are some talks from past year's Google I/O that hint at what we do and why. Search youtube.com for "app engine".

I'm adding an answer here because our current real-world results are the opposite of what we expect.

After sustained performance degradation, we tried switching our (Python) app back to NON-threadsafe mode and were very surprised to see our performance improved by about 10x. So we've left it off. Our GAE support team wasn't able to explain how this could be. The last time we profiled, we were quite I/O bound to the datastore and in theory should have still been getting a lot of lift from multi-threading.

So from our experience... not only don't assume threadsafe will be faster, it could be MUCH slower. If someone knows how this could be, please share.