I saw a discussion on a new framework at reddit. In this discussion,
someone ran a hello world test on this framework. He used Flask
configured with ngnix. I assume that this is a good set up as the
person appears to know what he's doing. This is on a VPS 512
Slicehost.
Seeing this, I thought it'd be interesting to look at a barebone
web2py's hello world. I'm on a 768 VPS, which is similar to Slicehost
512. I think Slicehost uses Zen and is more expensive. I'm with
Ramhost, which I like a lot. I have Apache, which is presumably
slower than nginx.
My barebone hello world is totally stripped down. Controller has only
1 function. No model. Compiled. Here's my result.
Server Software: Apache/2.2.16
Server Port: 80
Document Path: /hello/default/index
Document Length: 87 bytes
Concurrency Level: 100
Time taken for tests: 7.001 seconds
Complete requests: 2416
Failed requests: 0
Write errors: 0
Total transferred: 1181357 bytes
HTML transferred: 210453 bytes
Requests per second: 345.11 [#/sec] (mean)
Time per request: 289.759 [ms] (mean)
Time per request: 2.898 [ms] (mean, across all concurrent
requests)
Transfer rate: 164.80 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 52 68 11.0 65 144
Processing: 60 219 163.3 150 745
Waiting: 56 210 162.7 136 731
Total: 131 287 163.4 217 810
Percentage of the requests served within a certain time (ms)
50% 217
66% 298
75% 380
80% 443
90% 582
95% 649
98% 668
99% 711
100% 810 (longest request)
====
Observation:
Time per request for across connections is about 3ms for web2py, and
0.3ms for Flask. You can say Flask is 10x faster or you can say it's
3ms faster; it's pennies.
I would be very interested in a more serious comparison. Let's say a
blog with authentication, forms, texts and images. One uses web2py;
the other Flask + SQLAlchemy. The comparison metric will be:
(1) Efficiency/throughput: e.g. using Apache Benchmark
(2) Expressiveness: concise and understandable expression of codes.
I'm predicting that web2py will win (2), if DAL is faster than
SQLAlchemy. And depending on who you're asking, web2py will win (1)
as well.
This test will be very much needed because it will put things in
perspective. Real world apps are not "hello-worlds"'s. Let 's
compare non-trivial apps in meaningful ways.
=============
Reference:
Hello world on Flask + ngnix
http://www.reddit.com/r/Python/comments/gaegq/meinheld_is_a_highperformance_wsgicompliant_web/
=============
Ok, quick benchmark on my Linode 512:
flasktest.py
from flask import Flask, request, g, redirect, url_for, \
abort, render_template, flash
app = Flask(__name__)
@app.route('/')
def hello_world():
return "PONG"
if __name__ == '__main__':
app.run()
Running gunicorn with gevent worker, behind nginx:
Server Software: nginx/0.9.6
Server Hostname: flasktest.0xf.nl
Server Port: 80
Document Path: /
Document Length: 172 bytes
Concurrency Level: 100
Time taken for tests: 6.494 seconds
Complete requests: 20000
Failed requests: 19859
(Connect: 0, Receive: 0, Length: 19859, Exceptions: 0)
Write errors: 0
Non-2xx responses: 141
Total transferred: 3203124 bytes
HTML transferred: 103688 bytes
Requests per second: 3079.93 [#/sec] (mean)
Time per request: 32.468 [ms] (mean)
Time per request: 0.325 [ms] (mean, across all concurrent
requests)
Transfer rate: 481.71 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 4 4.4 3 22
Processing: 5 28 11.0 27 132
Waiting: 1 26 11.0 25 132
Total: 9 32 10.1 31 137
Percentage of the requests served within a certain time (ms)
50% 31
66% 35
75% 37
80% 38
90% 42
95% 47
98% 52
99% 57
100% 137 (longest request)
james@li140-209:/var/www/scripts$
