Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
JR Mayberry [EMAIL PROTECTED] wrote: The Modperl handler benchmark, which was done on a dual P3 500mhz on Linux does serious injustice to mod_perl. Anyone who uses Linux knows how horrible it is on SMP, I think some tests showed it uses as litle as 25% of the second processor.. It's an old post, but I simply cannot let this one pass uncommented. I run a busy Apache/mod_perl server on a 4-way SMP Linux box (kernel 2.2.13 from VA Linux), and it sure seems to be using all CPUs quite effectively. A simple benchmark with 'ab' shows the number of requests per second almost double when the concurrency is increased from 1 to 2. With a concurrency of 4, the number of requests per second increases to about 3.2 times the original, which is not bad at all considering that these are dynamic requests with DB queries. Anyway, I wouldn't expect the OS's SMP to be the limiting factor on Apache's dynamic page performance. Apache uses multiple processes, and dynamic page generation is generally CPU bound, not I/O bound. -- Roger Espel Llima, [EMAIL PROTECTED] http://www.iagora.com/~espel/index.html
[bordering on OT] Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
On Thu, 4 Jan 2001, Roger Espel Llima wrote: JR Mayberry [EMAIL PROTECTED] wrote: Linux does serious injustice to mod_perl. Anyone who uses Linux knows how horrible it is on SMP, I think some tests showed it uses as litle as 25% of the second processor.. A simple benchmark with 'ab' shows the number of requests per second almost double when the concurrency is increased from 1 to 2. With a concurrency of 4, the number of requests per second increases to about 3.2 times the original, which is not bad at all considering that these are dynamic requests with DB queries. Eh, ab isn't really made as anything other than the most coarsely-grained of benchmarks. Concurrency testing is useless because it will measure the ratio of requests/second/processor, not the scalability of requests from single to multiple processors. IOW, you would see almost exactly that same increase in req/second on a single processor, most likely, unless you have a really slow machine. You'd have to tune your load to give you one req/second/processor and then go from there for it to mean anything. Of course the original poster's statement on linux using only 25% of a second CPU is a fuddy and false generalization, but that's a different story. :P -- Blue Lang, Unix Voodoo Priest 202 Ashe Ave, Apt 3, Raleigh, NC. 919 835 1540 "I was born in a city of sharks and sailors!" - June of 44
Re: [bordering on OT] Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
On Thu, Jan 04, 2001 at 09:55:39AM -0500, Blue Lang wrote: Eh, ab isn't really made as anything other than the most coarsely-grained of benchmarks. Concurrency testing is useless because it will measure the ratio of requests/second/processor, not the scalability of requests from single to multiple processors. Yeah, I agree 'ab' is a pretty coarse benchmark. However, it does in a way measure how much the various processors are helping, because running ab with -c 1 should pretty much ensure that apache only uses one processor at a time (except for a slight overlap while one process does the logging and another could be reading the next request from another processor), and similarily -c 2 should let apache use 2 processors at one given time. All approximately, of course. Anyway, on that 4way server it works that way; the requests per second increase quickly with the concurrency up to 4, but don't increase anymore after that. That is serving relatively slow dynamic pages; with static content I'd expect more rapidly diminishing returns. -- Roger Espel Llima, [EMAIL PROTECTED] http://www.iagora.com/~espel/index.html
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
I strongly dislike the benchmarks on the below URL, its very misleading.. The Modperl handler benchmark, which was done on a dual P3 500mhz on Linux does serious injustice to mod_perl. Anyone who uses Linux knows how horrible it is on SMP, I think some tests showed it uses as litle as 25% of the second processor.. I don't think the benchmarks can be taken for anything, you have benchmarks running on different Architectures other then PC based... 300 mhz on Sun is totally different then 300 mhz on PC based products and the real number that drives this benchmark is HITS/S/MHZ.. I think for any real test that means anything, there needs to be some controlled factors, ie: the hardware and operating system.. Gunther Birznieks wrote: For the raw benchmarks... OK, I finally got a little time to download and read some the hello.tar.gz. It's good to see TT is fairly fast. But it's a shame that the only way to get faster than PHP is to write a raw Mod_perl handler according to the benchmarks. All the other mod_perl tools seem slower. JSP seems to also blow away mod_perl and PHP (except being almost equivalent to mod_perl handler speed). I assume Resin is precompiling JSP to Java classes and that maybe the JRE you are using does some very good hotspot on-the-fly machine-code compiling type technology? How does this benchmark stuff compare to the tests run at http://www.chamas.com/bench/ I notice that JSPs take quite a beating there but are running on a lower end machine on that set of tests. I presume the below tests are intended to replace the tests run on these various disparate machines. You also seem to have taken out tests? So you are no longer testing servlets only? It would be interesting to see if Servlet - JSP dispatching (with is the recommended model of coding Java Servlets/JSPs these days) results in any slow down. At 02:15 PM 12/16/2000 -0800, Joshua Chamas wrote: Hey, Still very rough, the hello world benchmark suite is available for download at: http://www.chamas.com/bench/hello.tar.gz You may run it like: # to get started, see what tests will run, note you # may need some CPAN modules installed to get this far perl ./bench.pl -test # to run tests for 1 minute ... shut down your programs # and walk away for best results. perl ./bench.pl -time=60 Here are my latest results, having added Resin/caucho/JSP with a J2RE 1.3.0 IBM java engine, which other benchmarks say is the fastest java on linux overall, from previous testing resin seems the fastest JSP. I changed the SSI tests to look more like the others, which also sped them considerably. Finally, I added tests for PHP, mine is 4.0.3, ePerl. Test Name Test File Hits/sec Total Hits Total Time sec/Hits Bytes/Hit -- -- -- -- -- -- Apache::ASP hello.asp 414.3 24857 hits 60.00 sec 0.002414 179 bytes Apache::Dispatch handler hello/worl 689.5 41375 hits 60.01 sec 0.001450 134 bytes Apache::Registry CGI Raw hello_raw. 725.2 43514 hits 60.00 sec 0.001379 52 bytes Apache::Registry CGI.pm hello.reg 491.5 29492 hits 60.00 sec 0.002035 154 bytes Apache::SSI hello.shtm 584.6 35080 hits 60.01 sec 0.001711 137 bytes Apache::ePerl hello.eper 359.8 21588 hits 60.00 sec 0.002780 155 bytes HTML static hello.html 1195.2 5 hits 41.83 sec 0.000837 249 bytes HTML::Embperl hello.epl 510.8 30647 hits 60.00 sec 0.001958 158 bytes HTML::Mason hello.mas 383.8 23030 hits 60.00 sec 0.002605 134 bytes Template Toolkit hello.tt553.6 33221 hits 60.01 sec 0.001806 136 bytes mod_caucho JSPhello.jsp 859.9 5 hits 58.15 sec 0.001163 156 bytes mod_include SSI hello.shtm 1008.0 5 hits 49.60 sec 0.000992 136 bytes mod_perl handler hello.benc 886.3 5 hits 56.42 sec 0.001128 134 bytes mod_php PHP hello.php 750.8 45050 hits 60.00 sec 0.001332 163 bytes As has been noted, my static html is probably slower than yours relatively. I have a dual CPU system have most apache modules enabled by default, thus creating huge headers for static html. I think the dual CPU nature of my system means my system will spend more time waiting on SMP network locking as the request rate gets faster, but I don't know much about these things, so if there is something to be gained here, please feel free to clarify how this might impact the results. --Josh _ Joshua Chamas Chamas Enterprises Inc. NodeWorks free web link monitoring Huntington Beach, CA USA http://www.nodeworks.com1-714-625-4051
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
JR Mayberry wrote: I strongly dislike the benchmarks on the below URL, its very misleading.. The Modperl handler benchmark, which was done on a dual P3 500mhz on Linux does serious injustice to mod_perl. Anyone who uses Linux knows how horrible it is on SMP, I think some tests showed it uses as litle as 25% of the second processor.. Please feel free to run the tests yourself, and if you give me the results, I'll be sure to post them at a later date at http://www.chamas.com/bench/ . You can grab the benchmarks from http://www.chamas.com/bench/hello.tar.gz Its only with data from multiple platforms that people will get a comfortable look at the environments cross platform exactly as you say. I don't think the benchmarks can be taken for anything, you have benchmarks running on different Architectures other then PC based... 300 mhz on Sun is totally different then 300 mhz on PC based products and the real number that drives this benchmark is HITS/S/MHZ.. Yes, I've done these 300Mhz tests before on Solaris WinNT and the results are at http://www.chamas.com/bench/ ... got to http://www.chamas.com/bench/hello_bysystem.html for anything useful. I think for any real test that means anything, there needs to be some controlled factors, ie: the hardware and operating system.. The new thing this round of testing is a portable test suite that anyone can run and control their factors as they please. --Josh _ Joshua Chamas Chamas Enterprises Inc. NodeWorks free web link monitoring Huntington Beach, CA USA http://www.nodeworks.com1-714-625-4051
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
On Mon, Dec 18, 2000 at 10:37:16AM -0800, Joshua Chamas wrote: Please feel free to run the tests yourself, and if you give me the results, I'll be sure to post them at a later date at http://www.chamas.com/bench/ . You can grab the benchmarks from http://www.chamas.com/bench/hello.tar.gz I downloaded the tar ball just to fool around on my system. It will be nice to have a README file with a short instruction set. Do you compile apache for each test or do you just use one size fits all apache with everything compiled in?
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
[EMAIL PROTECTED] wrote: On Mon, Dec 18, 2000 at 10:37:16AM -0800, Joshua Chamas wrote: Please feel free to run the tests yourself, and if you give me the results, I'll be sure to post them at a later date at http://www.chamas.com/bench/ . You can grab the benchmarks from http://www.chamas.com/bench/hello.tar.gz I downloaded the tar ball just to fool around on my system. It will be nice to have a README file with a short instruction set. Do you compile apache for each test or do you just use one size fits all apache with everything compiled in? One size fits all is the idea, but only the relevant parts of that benchmark will be loaded into a custom httpd.conf startup file for apache. The benchmarks get run at the same time on the same apache, so any overhead due to modules registering themselves into apache for handler stages will be passed to all the tests. Yes, I'll provide a README, and --help/-h help command line docs in the next release. -- Josh
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Hi all, On Sun, 17 Dec 2000, Gerald Richter wrote: there are so many factors, so they are very difficult to compare. True. But nevertheless I think it's a very useful bit of work because the thing that stands out is that all (server) dynamic content comes at a high cost in processor cycles. This won't come as a surprise to most of you. Be aware too that even the fastest of these benchmarks is not an outstandingly fast setup. The lesson is obvious: if you're looking for high throughput, think long and hard about your system architecture, in the widest possible sense, and keep server-generated dynamic stuff to a minimum. Having said that, most of us won't be pushing the limits most of the time (and if we are we're probably secretly quite pleased about it:). 73, Ged.
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
Gunther Birznieks wrote: But it's a shame that the only way to get faster than PHP is to write a raw Mod_perl handler according to the benchmarks. All the other mod_perl tools seem slower. It makes sense though. All the other tools do more setup work on each request: parsing input, manipulating it into objects, etc. JSP seems to also blow away mod_perl and PHP (except being almost equivalent to mod_perl handler speed). Hmm... To me that means it's equivalent, not blowing away mod_perl. I assume Resin is precompiling JSP to Java classes and that maybe the JRE you are using does some very good hotspot on-the-fly machine-code compiling type technology? The IBM JDK has a JIT, but no Hotspot equivalent. They have the fastest JVM on x86 though. Check out the scores here: http://www.volano.com/report.html One thing I discovered when testing Caucho's JSP last year is that they do a really good job of generating efficient servlets from JSP. I was not able to write a faster "hello world" servlet than the one they auto-generated from a JSP page. By the way, as a cautionary tale on "hello world" tests: These tests appear to show mod_include handling SSI pages twice as fast as Apache::SSI. However, in tests we've done with real production pages (30-60K, half dozen or so includes per page) Apache::SSI was actually a little bit faster. Of course this was about a year ago so the tables may have turned since then. - Perrin
Linux Hello World Benchmarks: +PHP,JSP,ePerl
Hey, Still very rough, the hello world benchmark suite is available for download at: http://www.chamas.com/bench/hello.tar.gz You may run it like: # to get started, see what tests will run, note you # may need some CPAN modules installed to get this far perl ./bench.pl -test # to run tests for 1 minute ... shut down your programs # and walk away for best results. perl ./bench.pl -time=60 Here are my latest results, having added Resin/caucho/JSP with a J2RE 1.3.0 IBM java engine, which other benchmarks say is the fastest java on linux overall, from previous testing resin seems the fastest JSP. I changed the SSI tests to look more like the others, which also sped them considerably. Finally, I added tests for PHP, mine is 4.0.3, ePerl. Test Name Test File Hits/sec Total Hits Total Time sec/Hits Bytes/Hit -- -- -- -- -- -- Apache::ASP hello.asp 414.3 24857 hits 60.00 sec 0.002414 179 bytes Apache::Dispatch handler hello/worl 689.5 41375 hits 60.01 sec 0.001450 134 bytes Apache::Registry CGI Raw hello_raw. 725.2 43514 hits 60.00 sec 0.001379 52 bytes Apache::Registry CGI.pm hello.reg 491.5 29492 hits 60.00 sec 0.002035 154 bytes Apache::SSI hello.shtm 584.6 35080 hits 60.01 sec 0.001711 137 bytes Apache::ePerl hello.eper 359.8 21588 hits 60.00 sec 0.002780 155 bytes HTML static hello.html 1195.2 5 hits 41.83 sec 0.000837 249 bytes HTML::Embperl hello.epl 510.8 30647 hits 60.00 sec 0.001958 158 bytes HTML::Mason hello.mas 383.8 23030 hits 60.00 sec 0.002605 134 bytes Template Toolkit hello.tt553.6 33221 hits 60.01 sec 0.001806 136 bytes mod_caucho JSPhello.jsp 859.9 5 hits 58.15 sec 0.001163 156 bytes mod_include SSI hello.shtm 1008.0 5 hits 49.60 sec 0.000992 136 bytes mod_perl handler hello.benc 886.3 5 hits 56.42 sec 0.001128 134 bytes mod_php PHP hello.php 750.8 45050 hits 60.00 sec 0.001332 163 bytes As has been noted, my static html is probably slower than yours relatively. I have a dual CPU system have most apache modules enabled by default, thus creating huge headers for static html. I think the dual CPU nature of my system means my system will spend more time waiting on SMP network locking as the request rate gets faster, but I don't know much about these things, so if there is something to be gained here, please feel free to clarify how this might impact the results. --Josh _ Joshua Chamas Chamas Enterprises Inc. NodeWorks free web link monitoring Huntington Beach, CA USA http://www.nodeworks.com1-714-625-4051
Re: Linux Hello World Benchmarks: +PHP,JSP,ePerl
For the raw benchmarks... OK, I finally got a little time to download and read some the hello.tar.gz. It's good to see TT is fairly fast. But it's a shame that the only way to get faster than PHP is to write a raw Mod_perl handler according to the benchmarks. All the other mod_perl tools seem slower. JSP seems to also blow away mod_perl and PHP (except being almost equivalent to mod_perl handler speed). I assume Resin is precompiling JSP to Java classes and that maybe the JRE you are using does some very good hotspot on-the-fly machine-code compiling type technology? How does this benchmark stuff compare to the tests run at http://www.chamas.com/bench/ I notice that JSPs take quite a beating there but are running on a lower end machine on that set of tests. I presume the below tests are intended to replace the tests run on these various disparate machines. You also seem to have taken out tests? So you are no longer testing servlets only? It would be interesting to see if Servlet - JSP dispatching (with is the recommended model of coding Java Servlets/JSPs these days) results in any slow down. At 02:15 PM 12/16/2000 -0800, Joshua Chamas wrote: Hey, Still very rough, the hello world benchmark suite is available for download at: http://www.chamas.com/bench/hello.tar.gz You may run it like: # to get started, see what tests will run, note you # may need some CPAN modules installed to get this far perl ./bench.pl -test # to run tests for 1 minute ... shut down your programs # and walk away for best results. perl ./bench.pl -time=60 Here are my latest results, having added Resin/caucho/JSP with a J2RE 1.3.0 IBM java engine, which other benchmarks say is the fastest java on linux overall, from previous testing resin seems the fastest JSP. I changed the SSI tests to look more like the others, which also sped them considerably. Finally, I added tests for PHP, mine is 4.0.3, ePerl. Test Name Test File Hits/sec Total Hits Total Time sec/Hits Bytes/Hit -- -- -- -- -- -- Apache::ASP hello.asp 414.3 24857 hits 60.00 sec 0.002414 179 bytes Apache::Dispatch handler hello/worl 689.5 41375 hits 60.01 sec 0.001450 134 bytes Apache::Registry CGI Raw hello_raw. 725.2 43514 hits 60.00 sec 0.001379 52 bytes Apache::Registry CGI.pm hello.reg 491.5 29492 hits 60.00 sec 0.002035 154 bytes Apache::SSI hello.shtm 584.6 35080 hits 60.01 sec 0.001711 137 bytes Apache::ePerl hello.eper 359.8 21588 hits 60.00 sec 0.002780 155 bytes HTML static hello.html 1195.2 5 hits 41.83 sec 0.000837 249 bytes HTML::Embperl hello.epl 510.8 30647 hits 60.00 sec 0.001958 158 bytes HTML::Mason hello.mas 383.8 23030 hits 60.00 sec 0.002605 134 bytes Template Toolkit hello.tt553.6 33221 hits 60.01 sec 0.001806 136 bytes mod_caucho JSPhello.jsp 859.9 5 hits 58.15 sec 0.001163 156 bytes mod_include SSI hello.shtm 1008.0 5 hits 49.60 sec 0.000992 136 bytes mod_perl handler hello.benc 886.3 5 hits 56.42 sec 0.001128 134 bytes mod_php PHP hello.php 750.8 45050 hits 60.00 sec 0.001332 163 bytes As has been noted, my static html is probably slower than yours relatively. I have a dual CPU system have most apache modules enabled by default, thus creating huge headers for static html. I think the dual CPU nature of my system means my system will spend more time waiting on SMP network locking as the request rate gets faster, but I don't know much about these things, so if there is something to be gained here, please feel free to clarify how this might impact the results. --Josh _ Joshua Chamas Chamas Enterprises Inc. NodeWorks free web link monitoring Huntington Beach, CA USA http://www.nodeworks.com1-714-625-4051 __ Gunther Birznieks ([EMAIL PROTECTED]) eXtropia - The Web Technology Company http://www.extropia.com/
