On Monday 28 September 2009 12:56:36 am Fayland Lam wrote: > I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8
Benchmark, as requested. View this message at http://p3m.org/pfn/3499 if your mailer is too high-tech for fixed-width text. The Setup: Linux 2.6 OpenVZ, on Quad 2.2GHz Opteron. Perl 5.10.0 + debian patches. FastCGI via Apache 2.2, mpm_event, mod_fastcgi. The two test instances were running on the same machine, with the same perl and the same checkout of the app, but different local::lib directories. FastCGI was set to 10 processes. The page I was hitting was from a real checkout of a real production app, however it was the front page of the site, which is fairly light on dynamic content. I figured this was appropriate since it would better show any differences in Catalyst rather than spending a lot of time in the backend. The code still hits several models, 3 actions, and a view, but perhaps it was a little too fast since, as you'll see below, my throughput was ultimately limited by the number of running processes. Each instance was given a "warmup" run (the results of which were discarded) before the following tests were run. My tool collects statistics on the return status, but for all tests the returns were all 200 (success) so I've left out that row. || 20 requests/second (20 threads) for 60s | Metric || Catalyst 5.7010 || Catalyst 5.8011 |===================||==========================||========================== | Hits || 1200 || 1200 | Throughput || 20.00 req/s || 20.00 req/s | Latency (mean) || 0.072s || 0.074s | Latency (SD) || 0.013s || 0.017s | Latency (Q1-Q3) || 0.064 - 0.078s || 0.066 - 0.080s || 40 requests/second (40 threads) for 60s | Metric || Catalyst 5.7010 || Catalyst 5.8011 |===================||==========================||========================== | Hits || 2400 || 2400 | Throughput || 40.00 req/s || 40.00 req/s | Latency (mean) || 0.083s || 0.088s | Latency (SD) || 0.020s || 0.024s | Latency (Q1-Q3) || 0.069 - 0.095s || 0.072 - 0.100s || 80 requests/second (80 threads) for 60s | Metric || Catalyst 5.7010 || Catalyst 5.8011 |===================||==========================||========================== | Hits || 4675 || 4637 | Throughput || 77.92 req/s || 77.28 req/s | Latency (mean) || 0.688s || 0.708s | Latency (SD) || 0.178s || 0.187s | Latency (Q1-Q3) || 0.617 - 0.800s || 0.726 - 0.811s The difference between 5.7 and 5.8 in these results is consistently in favor of 5.7, but by a margin of between 0% and 5% which is not a whole lot in my book. By my unscientific measure (i.e. looking at "top") of memory usage, 5.7 used 138MB of RAM (for fcgi-pm + 10x fcgi children) whereas 5.8 used 184MB, so that's a 33% expansion, which is a more significant issue. I have a feeling that most of that is shared, and so the difference wouldn't increase much with an increase in the number of processes, but I haven't investigated that yet. Questions? Andrew "hobbs" Rodland _______________________________________________ List: [email protected] Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/[email protected]/ Dev site: http://dev.catalyst.perl.org/
