Gunicorn

Performance

Performance can be measured in at least three ways:

• throughput (requests per second),
• latency (minimum),
• latency (average)

I use locust with a hatch rate so that in 10s all users are hatched to get some numbers. I use the following locustfile.py:

from locust import HttpLocust, TaskSet, task


class WebsiteTasks(TaskSet):
    @task
    def index(self):
        self.client.get("/")


class WebsiteUser(HttpLocust):
    task_set = WebsiteTasks
    min_wait = 5000
    max_wait = 15000

I had to execute ulimit -S -n 10240 in order to test the 10,000 users. Otherwise I got

[2019-05-26 12:25:03,499] pc07/ERROR/stderr: OSError: [Errno 16] Device or resource busy
[2019-05-26 12:25:03,499] pc07/ERROR/stderr: 
[2019-05-26 12:25:03,499] pc07/ERROR/stderr: 2019-05-26T10:25:03Z
[2019-05-26 12:25:03,499] pc07/ERROR/stderr: 
[2019-05-26 12:25:03,499] pc07/ERROR/stderr: (<ThreadPool at 0x7fc2e275d4a8 0/10/10 hub=<Hub at 0x7fc2e7c84780 thread_ident=0x140475181328192>>, <built-in function getaddrinfo>) failed with OSError

Constant Response

The following app.py is probably the simplest one possible.

from flask import Flask

app = Flask(__name__)


@app.route('/')
def hello():
    return 'hello world!'

Let's first have a look at the median response time by the number of users:

You can clearly see that there is no real difference up to 100 users. At 1000 users, the development server gets slow. gevent does not care about it. At 10,000 users, gevent gets slow. But the real difference was the 95-percentile response time. This varies over time. A lot.

I/O Bound

Let's say our workload was I/O bound. Maybe we need to request another API before we can answer. To simulate this, I let the request sleep for 1s before it answers:

import time

from flask import Flask

app = Flask(__name__)


@app.route('/')
def hello():
    time.sleep(1)
    return 'hello world!'

CPU Bound

Let's say our workload was CPU bound:

import time

from flask import Flask

app = Flask(__name__)


@app.route('/')
def hello():
    time.sleep(1)
    return 'hello world!'

Alternatives

• uWSGI (docs )is both a protocol and an application server; the application server can serve uWSGI, FastCGI, and HTTP protocols. Seems to be used together with gevent.
• Tornado: a scalable, non-blocking web server and web application framework

While I searched, I also found:

• TWISTED WEB
• CHERRYPY
• WAITRESS
• CHAUSSETTE
• meinheld: a high performance asynchronous WSGI Web Server (based on picoev)

Notes

• gevent_pywsgi and gevent_wsgi are the same.¹
• gevent uses the gunicorn event parser, while gevent_wsgi uses the event parser within itself.¹
• If you need to handle a high volume of concurrent requests and your application performs a lot of waiting on I/O (database, streaming responses, upstream requests, etc) then gevent can be a good choice.¹
• I didn't include Sanic, because it is less tested then Flask.[^2]

See also

• 2018-01-30: Brief introduction about the types of worker in gunicorn and respective suitable scenario
• 2015-01-20: Gunicorn Worker Types
• 2017-01-17: Reddit-comment by desmoulinmichel
• 2012-09-12: Benchmark uWSGI vs gunicorn for async workers by Ivan Dyedov

Sources