2010/3/24 Graham Dumpleton <[email protected]>: > Hmmmm, I think all this is showing is how hard it is to get reliable > results from one test run to the next. > > You can see for example how the max processing time and transfer rate > jump around so much. > > FWIW, theoretically, the AddHandler method should be slowest. The next > quickest should be WSGIScriptAlias. The fastest should be > WSGIQuickHandlerScript. > > Hang on. No, AddHandler inside of .htaccess file would definitely be > slower. If AddHandler in main Apache configuration then you are > avoiding .htaccess parsing and so possibly no real difference to > WSGIScriptAlias or if any difference only ever so slightly slower. > > WSGIQuickHandlerScript should be quickest because it is using a > special hook in Apache that gets to run and is intended to circumvent > the normal request processing phases. > > You can see a depiction of the Apache request phases in: > > http://www.kuzbass.ru/docs/apache_c_mod_perl/24.htm#BIN68 > > The ap_quick_handler_hook() comes in even before the post read request phase. > > The thought in trying this was that the attitude of many Python web > developers is that everything should be done in the WSGI application > and as such they don't care about what Apache does or can do for them. > As such, one could hook in so early that all the extra stuff that > Apache does is skipped. > > By doing this though you are even skipping normal Apache URL matching > and so this can only be used in a VirtualHost where absolutely > everything is being handled by the WSGI application. That is, can't > even serve static files out of the same VirtualHost and would need to > use a different VirtualHost for that. > > Theoretically though, skipping all those extra phases should be that > the overhead of Apache request processing is less. In the context of a > hello world program this may be noticeable, but still insignificant in > the context of a fat Python web application where one is talking about > 100s of request/sec vs 1000s for a hello world application. > > So, there may be some use for this with applications which are very > very streamlined and optimised that really need to push for maximum > performance, but in general possibly not worth it. > > Anyway, still have some issues with this to sort out, as possible that > by hooking in this early, even the cleanup of REQUEST_URI hasn't been > done and so things like repeating slashes and encoded slashes still > occur in PATH_INFO. At the same time, some people actually have wanted > that.
One of the main advantages of using mod_wsgi is to have Apache doing all the Apachy stuff. If people don't need it then they could use other servers or their own or even one of the mod_python handlers (low coolness factor I know). IMHO if it will add to mod_wsgi complexity then it is not worth. Clodoaldo > > Graham > > On 24 March 2010 18:48, Clodoaldo Neto <[email protected]> wrote: >> 2010/3/24 Clodoaldo Neto <[email protected]>: >>> 2010/3/23 Graham Dumpleton <[email protected]>: >>>> For whatever reason I often have lots of problems with doing benchmark >>>> testing on MacOS X. There are some known issues with MacOS X and >>>> Apache, so possibly I am always hitting that when trying to stress >>>> test stuff. >>>> >>>> As such, wandering if any eager people out there who would like to run >>>> some tests for me on a non MacOS X platform which may give more >>>> repeatable results. UNIX systems preferred as Windows is usually worse >>>> than MacOS X for predictability. >>>> >>>> All I need for the test is a comparative set of 'ab' output for a few >>>> different mod_wsgi configurations. The intent here is to compare the >>>> different configurations so consistency of results is more important >>>> than outright speed. >>>> >>>> Note that you MUST being using latest mod_wsgi from subversion trunk >>>> to do this test. >>>> >>>> The hello world program should be that as detailed on: >>>> >>>> http://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide >>>> >>>> Call the file 'hello.wsgi' and stick it in '/some/directory'. >>>> >>>> The first configuration is: >>>> >>>> Alias / /some/directory/ >>>> >>>> <Directory /some/directory> >>>> AddHandler wsgi-script .wsgi >>>> Order Deny,Allow >>>> Allow from All >>>> </Directory> >>>> >>>> The second configuration is: >>>> >>>> WSGIScriptAlias / /some/directory/hello.wsgi >>>> >>>> <Directory /some/directory> >>>> Order Deny,Allow >>>> Allow from All >>>> </Directory> >>>> >>>> The third configuration is: >>>> >>>> WSGIQuickHandlerScript /some/directory/hello.wsgi >>>> >>>> In all cases use URL on the site as '/hello.wsgi'. >>>> >>>> Am happy with a test such as: >>>> >>>> ab -n 10000 http://localhost/hello.wsgi >>>> >>>> The more requests the merrier. Anything below 5000 isn't particularly >>>> useful as too variable with respect to Apache start up costs. >>>> >>>> If you want to try different levels of concurrency or try different >>>> tools, that is your call. >>>> >>>> Average results from some number of runs if that makes you feel >>>> happier that getting good results. >>>> >>>> If using 'ab', always keep an eye on the section: >>>> >>>> Connection Times (ms) >>>> min mean[+/-sd] median max >>>> Connect: 0 0 0.1 0 6 >>>> Processing: 0 0 0.2 0 7 >>>> Waiting: 0 0 0.2 0 7 >>>> Total: 0 0 0.2 0 7 >>>> >>>> From one run to the next, you want to get similar results for that >>>> 'max' column and discard anything that shows some out of the ordinary >>>> happened. For example, crap like I get on MacOS X such as: >>>> >>>> Connection Times (ms) >>>> min mean[+/-sd] median max >>>> Connect: 0 2 126.6 0 6935 >>>> Processing: 0 0 0.3 0 10 >>>> Waiting: 0 0 0.3 0 10 >>>> Total: 0 3 126.6 0 6936 >>>> >>>> Not sure if the DNS here simply sucks or what the issue is. :-( >>>> >>>> Thanks in advance. >>>> >>>> Will explain what this is about when I see some results and know if it >>>> makes much difference or not. >>>> >>>> Graham >>> >>> Alias / /home/www/default/ >>> <Directory /home/www/default> >>> Options +ExecCGI >>> AddHandler wsgi-script .wsgi >>> Order Deny,Allow >>> Allow from All >>> </Directory> >>> >>> $ ab -n 100000 http://10.1.1.110/hello.wsgi >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking 10.1.1.110 (be patient) >>> Completed 10000 requests >>> Completed 20000 requests >>> Completed 30000 requests >>> Completed 40000 requests >>> Completed 50000 requests >>> Completed 60000 requests >>> Completed 70000 requests >>> Completed 80000 requests >>> Completed 90000 requests >>> Completed 100000 requests >>> Finished 100000 requests >>> >>> >>> Server Software: Apache/2.2.14 >>> Server Hostname: 10.1.1.110 >>> Server Port: 80 >>> >>> Document Path: /hello.wsgi >>> Document Length: 12 bytes >>> >>> Concurrency Level: 1 >>> Time taken for tests: 50.235 seconds >>> Complete requests: 100000 >>> Failed requests: 0 >>> Write errors: 0 >>> Total transferred: 18000000 bytes >>> HTML transferred: 1200000 bytes >>> Requests per second: 1990.66 [#/sec] (mean) >>> Time per request: 0.502 [ms] (mean) >>> Time per request: 0.502 [ms] (mean, across all concurrent requests) >>> Transfer rate: 349.92 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.0 0 6 >>> Processing: 0 0 0.4 0 38 >>> Waiting: 0 0 0.3 0 25 >>> Total: 0 0 0.4 0 38 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 0 >>> 66% 0 >>> 75% 1 >>> 80% 1 >>> 90% 1 >>> 95% 1 >>> 98% 1 >>> 99% 1 >>> 100% 38 (longest request) >>> ------------------------------------------------- >>> >>> WSGIScriptAlias / /home/www/default/ >>> <Directory /home/www/default> >>> Options +ExecCGI >>> Order Deny,Allow >>> Allow from All >>> </Directory> >>> >>> $ ab -n 100000 http://10.1.1.110/hello.wsgi >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking 10.1.1.110 (be patient) >>> Completed 10000 requests >>> Completed 20000 requests >>> Completed 30000 requests >>> Completed 40000 requests >>> Completed 50000 requests >>> Completed 60000 requests >>> Completed 70000 requests >>> Completed 80000 requests >>> Completed 90000 requests >>> Completed 100000 requests >>> Finished 100000 requests >>> >>> >>> Server Software: Apache/2.2.14 >>> Server Hostname: 10.1.1.110 >>> Server Port: 80 >>> >>> Document Path: /hello.wsgi >>> Document Length: 12 bytes >>> >>> Concurrency Level: 1 >>> Time taken for tests: 50.152 seconds >>> Complete requests: 100000 >>> Failed requests: 0 >>> Write errors: 0 >>> Total transferred: 18000000 bytes >>> HTML transferred: 1200000 bytes >>> Requests per second: 1993.96 [#/sec] (mean) >>> Time per request: 0.502 [ms] (mean) >>> Time per request: 0.502 [ms] (mean, across all concurrent requests) >>> Transfer rate: 350.50 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.0 0 7 >>> Processing: 0 0 2.1 0 645 >>> Waiting: 0 0 2.1 0 645 >>> Total: 0 0 2.1 0 645 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 0 >>> 66% 0 >>> 75% 0 >>> 80% 1 >>> 90% 1 >>> 95% 1 >>> 98% 1 >>> 99% 1 >>> 100% 645 (longest request) >>> --------------------------------------------------------- >>> >>> WSGIQuickHandlerScript /home/www/default/ >>> <Directory /home/www/default> >>> Options +ExecCGI >>> Order Deny,Allow >>> Allow from All >>> </Directory> >>> >>> $ ab -n 100000 http://10.1.1.110/hello.wsgi >>> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >>> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >>> Licensed to The Apache Software Foundation, http://www.apache.org/ >>> >>> Benchmarking 10.1.1.110 (be patient) >>> Completed 10000 requests >>> Completed 20000 requests >>> Completed 30000 requests >>> Completed 40000 requests >>> Completed 50000 requests >>> Completed 60000 requests >>> Completed 70000 requests >>> Completed 80000 requests >>> Completed 90000 requests >>> Completed 100000 requests >>> Finished 100000 requests >>> >>> >>> Server Software: Apache/2.2.14 >>> Server Hostname: 10.1.1.110 >>> Server Port: 80 >>> >>> Document Path: /hello.wsgi >>> Document Length: 284 bytes >>> >>> Concurrency Level: 1 >>> Time taken for tests: 36.570 seconds >>> Complete requests: 100000 >>> Failed requests: 0 >>> Write errors: 0 >>> Non-2xx responses: 100000 >>> Total transferred: 46400000 bytes >>> HTML transferred: 28400000 bytes >>> Requests per second: 2734.49 [#/sec] (mean) >>> Time per request: 0.366 [ms] (mean) >>> Time per request: 0.366 [ms] (mean, across all concurrent requests) >>> Transfer rate: 1239.07 [Kbytes/sec] received >>> >>> Connection Times (ms) >>> min mean[+/-sd] median max >>> Connect: 0 0 0.0 0 4 >>> Processing: 0 0 2.8 0 882 >>> Waiting: 0 0 2.8 0 882 >>> Total: 0 0 2.8 0 882 >>> >>> Percentage of the requests served within a certain time (ms) >>> 50% 0 >>> 66% 0 >>> 75% 0 >>> 80% 0 >>> 90% 0 >>> 95% 0 >>> 98% 1 >>> 99% 1 >>> 100% 882 (longest request) >>> >>> Regards, Clodoaldo >>> >> >> This second and third were wrongly configured. Forgot the script name. >> >> WSGIScriptAlias / /home/www/default/hello.wsgi >> <Directory /home/www/default> >> Options +ExecCGI >> Order Deny,Allow >> Allow from All >> </Directory> >> >> $ ab -n 100000 http://10.1.1.110/hello.wsgi >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking 10.1.1.110 (be patient) >> Completed 10000 requests >> Completed 20000 requests >> Completed 30000 requests >> Completed 40000 requests >> Completed 50000 requests >> Completed 60000 requests >> Completed 70000 requests >> Completed 80000 requests >> Completed 90000 requests >> Completed 100000 requests >> Finished 100000 requests >> >> >> Server Software: Apache/2.2.14 >> Server Hostname: 10.1.1.110 >> Server Port: 80 >> >> Document Path: /hello.wsgi >> Document Length: 12 bytes >> >> Concurrency Level: 1 >> Time taken for tests: 59.074 seconds >> Complete requests: 100000 >> Failed requests: 0 >> Write errors: 0 >> Total transferred: 18000000 bytes >> HTML transferred: 1200000 bytes >> Requests per second: 1692.78 [#/sec] (mean) >> Time per request: 0.591 [ms] (mean) >> Time per request: 0.591 [ms] (mean, across all concurrent requests) >> Transfer rate: 297.56 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.0 0 2 >> Processing: 0 1 2.9 0 902 >> Waiting: 0 0 2.9 0 901 >> Total: 0 1 2.9 0 902 >> >> Percentage of the requests served within a certain time (ms) >> 50% 0 >> 66% 1 >> 75% 1 >> 80% 1 >> 90% 1 >> 95% 1 >> 98% 1 >> 99% 1 >> 100% 902 (longest request) >> --------------------------------------------------------------------- >> >> WSGIQuickHandlerScript /home/www/default/hello.wsgi >> <Directory /home/www/default> >> Options +ExecCGI >> Order Deny,Allow >> Allow from All >> </Directory> >> >> $ ab -n 100000 http://10.1.1.110/hello.wsgi >> This is ApacheBench, Version 2.3 <$Revision: 655654 $> >> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ >> Licensed to The Apache Software Foundation, http://www.apache.org/ >> >> Benchmarking 10.1.1.110 (be patient) >> Completed 10000 requests >> Completed 20000 requests >> Completed 30000 requests >> Completed 40000 requests >> Completed 50000 requests >> Completed 60000 requests >> Completed 70000 requests >> Completed 80000 requests >> Completed 90000 requests >> Completed 100000 requests >> Finished 100000 requests >> >> >> Server Software: Apache/2.2.14 >> Server Hostname: 10.1.1.110 >> Server Port: 80 >> >> Document Path: /hello.wsgi >> Document Length: 12 bytes >> >> Concurrency Level: 1 >> Time taken for tests: 46.400 seconds >> Complete requests: 100000 >> Failed requests: 0 >> Write errors: 0 >> Total transferred: 18000000 bytes >> HTML transferred: 1200000 bytes >> Requests per second: 2155.16 [#/sec] (mean) >> Time per request: 0.464 [ms] (mean) >> Time per request: 0.464 [ms] (mean, across all concurrent requests) >> Transfer rate: 378.84 [Kbytes/sec] received >> >> Connection Times (ms) >> min mean[+/-sd] median max >> Connect: 0 0 0.0 0 4 >> Processing: 0 0 0.4 0 72 >> Waiting: 0 0 0.4 0 71 >> Total: 0 0 0.4 0 72 >> >> Percentage of the requests served within a certain time (ms) >> 50% 0 >> 66% 0 >> 75% 0 >> 80% 0 >> 90% 1 >> 95% 1 >> 98% 1 >> 99% 1 >> 100% 72 (longest request) >> >> Sorry for the noise, Clodoaldo >> >> -- >> You received this message because you are subscribed to the Google Groups >> "modwsgi" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/modwsgi?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
