2009/1/11 Tycon <[email protected]>: > > well I'm a performance person and I hate wasting time because of bad/ > sloppy design and implementation. And even for low volume apps, a well > tuned app will have a better response time, as well as higher load > capacity and better sclalability. > > In pure request per second I made a bunch of tune-up that resulted in > these improvements: > 1. Use python2.6 ==> +5%
And maybe a 20% in near future: http://bugs.python.org/issue4753 > 2. Use memcached to store sessions ==> +10% I agree. > 3. Make sure session is loaded and saved only once instead of multiple > time during a request ==> 5-10% I agree. > 4. Use cherryPy instead of paste HTTP server ==> 5-10% Have you tested spawn? I don't do test still :-/ It's this number valid with a external cache system? > 5. Use nginx as a proxy instead of apache => 25% (and save 15MB per > worker) I use nginx too. For proxy and static serve. > 6. Remove redunandant code from middleware (e.g. cache middleware) Or SessionMidleware if you don't use it. It saves a coockie and cache is easier. > 7. Use mutiple processes instead of just increasing the number of > threads in a single paster app server => 15-20% Obvious. Python have a Global Interpreter Lock, all threads run in only one core. This diference can increase with more cores. > 8. Cache pages in memcached, and have nginx bypass app server by > fetching directly from memcached (and use ssi to render dynamic > fragments) ==> 150% Varnish + ESI is another interesting option. The cache is stored in disk (cached disk space must be only a bit less speeder than memcache). I have not tested yet disk latency versus net latencies (memcached limited the space about half memory dedicated and you need more machines for same cache space). Net and memory bandwidth is wider than disk bandwidth, but this could be secondary in an hosted environment (only one net interface, limited ram, a lot of disk space, another clients competing for same net resources, ...). You can purge the cache and create complex config with VCL (a domain specific language that varnish compiles to C). For example. The next is a test with a slow application that render a very complex xslt runned in a local machine. This attach directly a pylons server: $ http_load -parallel 5 -seconds 5 l_box.uri 959 fetches, 5 max parallel, 4.06616e+06 bytes, in 5.00126 seconds 4240 mean bytes/connection 191.752 fetches/sec, 813027 bytes/sec msecs/connect: 0.228164 mean, 5.447 max, 0.028 min msecs/first-response: 24.1095 mean, 65.136 max, 12.562 min HTTP response codes: code 200 -- 959 And the same test runned in other machine. This use a nginx+varnish front-end: $ http_load -parallel 5 -seconds 5 box.url 11224 fetches, 5 max parallel, 4.75898e+07 bytes, in 5 seconds 4240 mean bytes/connection 2244.8 fetches/sec, 9.51795e+06 bytes/sec msecs/connect: 1.38112 mean, 3011.76 max, 0.09 min msecs/first-response: 0.530991 mean, 16.104 max, 0.316 min HTTP response codes: code 200 -- 11224 The two machines are in the same net of the hosting. If I wan't scale up I haven't got available machines in these net at this moment. From another machine in the same hosting, different net, I obtain 277 fetches/sec. A esi or ssi use make a more complex development environment. This is important (production time vs development time). Maybe a good helpers and an mock midleware can help this issue. > > With all these small fixes and optimizations I end up with something > that's X-times faster and is noticeable even for low volume apps due > to much better response time improving the user experience. > YSlow firefox plugin helps a lot too. > Not to mention that I keep a scalable upgrade path by making sure each > component can easily be move to a different machine (e.g. => dont use > mod_wsgi or local caches). > This rule discards Varnish, but I still need practical results ;-) Excuse my poor english. Regards, Javi > > On Jan 11, 1:02 pm, "Mike Orr" <[email protected]> wrote: >> Does Tycon's application actually *need* to be ultra-conservative on >> overhead, or is this more of an aesthetic desire? I.e., would it >> actually not run, or require more servers, if all these steps weren't >> taken? Some applications are so high-load that they require all these >> steps, but many applications don't. >> >> -- >> Mike Orr <[email protected]> > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "pylons-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/pylons-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
