Re: [Catalyst] Re: Catalyst benchmark 5.7 VS 5.8
Fayland Lam wrote: Tomas Doran wrote: top totally doesn't show how much RAM is shared by copy on write at all, and so is misleading you here. do you know how to do a real benchmark? the siege result shows 5.7 is better under pressure. I didn't actually do any 'real' benchmarking for this, I just pointed out that the conclusions you were drawing from top were rubbish. Andrew and Tobys memory benchmarks make sense and are much nearer what I'd expect. 1.5Mb more per process is much more reasonable than the 20Mb you were speculating initially. Cheers t0m ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Catalyst benchmark 5.7 VS 5.8
Tomas Doran wrote: Fayland Lam wrote: from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7 No, that'll be 20Mb of RAM _in total_, as all of those pages should be shared between your apache processes (given that you're preloading your application in the parent process). top totally doesn't show how much RAM is shared by copy on write at all, and so is misleading you here. do you know how to do a real benchmark? the siege result shows 5.7 is better under pressure. Thanks. Cheers t0m ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Catalyst benchmark 5.7 VS 5.8
Toby Corkindale wrote: (Apologies for top-posting.. have momentarily lost the option to change quoting styles it seems..) Fayland, I was looking at the benchmarks that you linked, and was just wondering which version of Perl you're running against? (CentOS 5 was one of the operating systems that came with the badly-patched Perl with the slow bless performance.. although I'm sure it's been patched by now? ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/ ) Thanks for your update. but it doesn't help on the benchmark since they are run on the same condition. so 5.7 is really better than 5.8 under siege. Thanks. Cheers, Toby - Original Message - From: Fayland Lam fayl...@gmail.com To: catalyst@lists.scsys.co.uk Sent: Mon, 28 Sep 2009 15:56:36 +1000 (EST) Subject: [Catalyst] Catalyst benchmark 5.7 VS 5.8 I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8 here is the result from our server: http://scsys.co.uk:8001/34323 the background is Catalyst 5.7011 VS Catalyst 5.80013 CPU: Intel(R) Core(TM)2 Quad CPU Q8200 @ 2.33GHz RAM: 4G OS: Centos5 from the top, each httpd takes 20M more RAM in 5.8 compared with 5.7 5.7 PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 22979 apache16 0 167m 142m 4248 S 17.0 3.5 0:06.07 httpd 5.8 PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 24813 apache15 0 190m 165m 4000 S 15.6 4.1 0:02.56 httpd in this case, I really can't let my boss agree me to upgrade the Catalyst. is it normal? any thoughts? Thanks. ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
Re: [Catalyst] Re: Catalyst benchmark 5.7 VS 5.8
On Monday 28 September 2009 09:31:13 pm Fayland Lam wrote: Toby Corkindale wrote: Fayland, I was looking at the benchmarks that you linked, and was just wondering which version of Perl you're running against? (CentOS 5 was one of the operating systems that came with the badly-patched Perl with the slow bless performance.. although I'm sure it's been patched by now? ie. http://blog.vipul.net/2008/08/24/redhat-perl-what-a-tragedy/ ) Thanks for your update. but it doesn't help on the benchmark since they are run on the same condition. so 5.7 is really better than 5.8 under siege. That doesn't follow. If your perl is broken and very slow at doing something that 5.8 does much more often, then 5.8 can be slower for you without being slower for everyone else. I have a real app and a suitable test system at hand -- I'll run some benches of my own and at least add a few more data points to the mix. Andrew ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
[Catalyst] Re: Catalyst benchmark 5.7 VS 5.8
Thanks for that. (Toby Corkindale too) I'll do more research and update you later. Thanks. Andrew Rodland wrote: On Monday 28 September 2009 12:56:36 am Fayland Lam wrote: I'm wondering if someone here did a benchmark between Catalyst 5.7 and 5.8 Benchmark, as requested. View this message at http://p3m.org/pfn/3499 if your mailer is too high-tech for fixed-width text. The Setup: Linux 2.6 OpenVZ, on Quad 2.2GHz Opteron. Perl 5.10.0 + debian patches. FastCGI via Apache 2.2, mpm_event, mod_fastcgi. The two test instances were running on the same machine, with the same perl and the same checkout of the app, but different local::lib directories. FastCGI was set to 10 processes. The page I was hitting was from a real checkout of a real production app, however it was the front page of the site, which is fairly light on dynamic content. I figured this was appropriate since it would better show any differences in Catalyst rather than spending a lot of time in the backend. The code still hits several models, 3 actions, and a view, but perhaps it was a little too fast since, as you'll see below, my throughput was ultimately limited by the number of running processes. Each instance was given a warmup run (the results of which were discarded) before the following tests were run. My tool collects statistics on the return status, but for all tests the returns were all 200 (success) so I've left out that row. || 20 requests/second (20 threads) for 60s | Metric|| Catalyst 5.7010 || Catalyst 5.8011 |===||==||== | Hits || 1200 || 1200 | Throughput|| 20.00 req/s || 20.00 req/s | Latency (mean)|| 0.072s || 0.074s | Latency (SD) || 0.013s || 0.017s | Latency (Q1-Q3) || 0.064 - 0.078s || 0.066 - 0.080s || 40 requests/second (40 threads) for 60s | Metric|| Catalyst 5.7010 || Catalyst 5.8011 |===||==||== | Hits || 2400 || 2400 | Throughput|| 40.00 req/s || 40.00 req/s | Latency (mean)|| 0.083s || 0.088s | Latency (SD) || 0.020s || 0.024s | Latency (Q1-Q3) || 0.069 - 0.095s || 0.072 - 0.100s || 80 requests/second (80 threads) for 60s | Metric|| Catalyst 5.7010 || Catalyst 5.8011 |===||==||== | Hits || 4675 || 4637 | Throughput|| 77.92 req/s || 77.28 req/s | Latency (mean)|| 0.688s || 0.708s | Latency (SD) || 0.178s || 0.187s | Latency (Q1-Q3) || 0.617 - 0.800s || 0.726 - 0.811s The difference between 5.7 and 5.8 in these results is consistently in favor of 5.7, but by a margin of between 0% and 5% which is not a whole lot in my book. By my unscientific measure (i.e. looking at top) of memory usage, 5.7 used 138MB of RAM (for fcgi-pm + 10x fcgi children) whereas 5.8 used 184MB, so that's a 33% expansion, which is a more significant issue. I have a feeling that most of that is shared, and so the difference wouldn't increase much with an increase in the number of processes, but I haven't investigated that yet. Questions? Andrew hobbs Rodland ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/ ___ List: Catalyst@lists.scsys.co.uk Listinfo: http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/catalyst Searchable archive: http://www.mail-archive.com/catalyst@lists.scsys.co.uk/ Dev site: http://dev.catalyst.perl.org/
