On Mon, Mar 22, 2010 at 11:23 PM, Graham Dumpleton
<[email protected]> wrote:
> For whatever reason I often have lots of problems with doing benchmark
> testing on MacOS X. There are some known issues with MacOS X and
> Apache, so possibly I am always hitting that when trying to stress
> test stuff.
>
> As such, wandering if any eager people out there who would like to run
> some tests for me on a non MacOS X platform which may give more
> repeatable results. UNIX systems preferred as Windows is usually worse
> than MacOS X for predictability.
>
> All I need for the test is a comparative set of 'ab' output for a few
> different mod_wsgi configurations. The intent here is to compare the
> different configurations so consistency of results is more important
> than outright speed.
>
> Note that you MUST being using latest mod_wsgi from subversion trunk
> to do this test.
>
> The hello world program should be that as detailed on:
>
> http://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide
>
> Call the file 'hello.wsgi' and stick it in '/some/directory'.
These tests are being run on a virtual server hosted on RackSpace w/
256MB memory running Ubuntu 9.10. Apache2 was installed via apt-get.
The default config file was modified, apache2 restarted and then the
test was run. I can rerun these longer or under different configs if
needed.
> The first configuration is:
>
> Alias / /some/directory/
>
> <Directory /some/directory>
> AddHandler wsgi-script .wsgi
> Order Deny,Allow
> Allow from All
> </Directory>
r...@wsgiperf:~# cat /etc/apache2/sites-enabled/000-default
<VirtualHost *:80>
ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined
Alias / /var/www/
<Directory /var/www>
AddHandler wsgi-script .wsgi
Order Deny,Allow
Allow from All
</Directory>
</VirtualHost>
r...@wsgiperf:~# ab -n 10000 http://localhost/hello.wsgi
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests
Server Software: Apache/2.2.12
Server Hostname: localhost
Server Port: 80
Document Path: /hello.wsgi
Document Length: 12 bytes
Concurrency Level: 1
Time taken for tests: 3.200 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1880000 bytes
HTML transferred: 120000 bytes
Requests per second: 3125.00 [#/sec] (mean)
Time per request: 0.320 [ms] (mean)
Time per request: 0.320 [ms] (mean, across all concurrent requests)
Transfer rate: 573.73 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.4 0 10
Processing: 0 0 1.7 0 10
Waiting: 0 0 1.5 0 10
Total: 0 0 1.7 0 10
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 10
99% 10
100% 10 (longest request)
r...@wsgiperf:~#
> The second configuration is:
>
> WSGIScriptAlias / /some/directory/hello.wsgi
>
> <Directory /some/directory>
> Order Deny,Allow
> Allow from All
> </Directory>
r...@wsgiperf:~# cat /etc/apache2/sites-enabled/000-default
<VirtualHost *:80>
ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined
WSGIScriptAlias / /var/www/hello.wsgi
<Directory /var/www>
Order Deny,Allow
Allow from All
</Directory>
</VirtualHost>
r...@wsgiperf:~# ab -n 10000 http://localhost/hello.wsgi
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests
Server Software: Apache/2.2.12
Server Hostname: localhost
Server Port: 80
Document Path: /hello.wsgi
Document Length: 12 bytes
Concurrency Level: 1
Time taken for tests: 3.430 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1880000 bytes
HTML transferred: 120000 bytes
Requests per second: 2915.45 [#/sec] (mean)
Time per request: 0.343 [ms] (mean)
Time per request: 0.343 [ms] (mean, across all concurrent requests)
Transfer rate: 535.26 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 10
Processing: 0 0 1.8 0 20
Waiting: 0 0 1.6 0 20
Total: 0 0 1.8 0 20
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 10
99% 10
100% 20 (longest request)
r...@wsgiperf:~#
> The third configuration is:
>
> WSGIQuickHandlerScript /some/directory/hello.wsgi
r...@wsgiperf:~# /etc/init.d/apache2 restart
* Restarting web server apache2
apache2: Could not reliably determine the server's fully qualified
domain name, using 173.203.113.220 for ServerName
... waiting .apache2: Could not reliably determine the server's fully
qualified domain name, using 173.203.113.220 for ServerName
...done.
r...@wsgiperf:~# cat /etc/apache2/sites-enabled/000-default
<VirtualHost *:80>
ErrorLog /var/log/apache2/error.log
LogLevel warn
CustomLog /var/log/apache2/access.log combined
WSGIQuickHandlerScript /var/www/hello.wsgi
</VirtualHost>
r...@wsgiperf:~# ab -n 10000 http://localhost/hello.wsgi
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking localhost (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests
Server Software: Apache/2.2.12
Server Hostname: localhost
Server Port: 80
Document Path: /hello.wsgi
Document Length: 12 bytes
Concurrency Level: 1
Time taken for tests: 3.330 seconds
Complete requests: 10000
Failed requests: 0
Write errors: 0
Total transferred: 1880000 bytes
HTML transferred: 120000 bytes
Requests per second: 3003.00 [#/sec] (mean)
Time per request: 0.333 [ms] (mean)
Time per request: 0.333 [ms] (mean, across all concurrent requests)
Transfer rate: 551.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.3 0 10
Processing: 0 0 1.7 0 10
Waiting: 0 0 1.6 0 10
Total: 0 0 1.7 0 10
Percentage of the requests served within a certain time (ms)
50% 0
66% 0
75% 0
80% 0
90% 0
95% 0
98% 10
99% 10
100% 10 (longest request)
r...@wsgiperf:~#
> In all cases use URL on the site as '/hello.wsgi'.
>
> Am happy with a test such as:
>
> ab -n 10000 http://localhost/hello.wsgi
>
> The more requests the merrier. Anything below 5000 isn't particularly
> useful as too variable with respect to Apache start up costs.
>
> If you want to try different levels of concurrency or try different
> tools, that is your call.
>
> Average results from some number of runs if that makes you feel
> happier that getting good results.
>
> If using 'ab', always keep an eye on the section:
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 0 0.1 0 6
> Processing: 0 0 0.2 0 7
> Waiting: 0 0 0.2 0 7
> Total: 0 0 0.2 0 7
>
> From one run to the next, you want to get similar results for that
> 'max' column and discard anything that shows some out of the ordinary
> happened. For example, crap like I get on MacOS X such as:
>
> Connection Times (ms)
> min mean[+/-sd] median max
> Connect: 0 2 126.6 0 6935
> Processing: 0 0 0.3 0 10
> Waiting: 0 0 0.3 0 10
> Total: 0 3 126.6 0 6936
>
> Not sure if the DNS here simply sucks or what the issue is. :-(
>
> Thanks in advance.
>
> Will explain what this is about when I see some results and know if it
> makes much difference or not.
>
> Graham
>
> --
> You received this message because you are subscribed to the Google Groups
> "modwsgi" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/modwsgi?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en.
