Re: Performance numbers, HELP!
Hi, At 5:37 AM +0200 4/19/02, Gerald Richter wrote: > > Any idea on what is going on and why the Embperl numbers are so low? >> I added debugging to make sure the pages were being cached and after >> the warm up the all showed 100% cache hits. The surprise to the > > group was the SPP numbers and how it out performed them all. > >Of course a simple module is faster than a more complex, but when you use it >you find yourself adding all these features that the more complex ones >already have and at the end your resulting application will not be much >faster, but you spend much more time developing it. I completely agree. The SPP was done to see the difference between the more mature template systems and a plain striped down one. > > Side Note: I wanted to try to use Embperl 2.0, but I couldn't get it > > to compile and didn't have the time to spend on it. > >If you really want to benchmark, you should use 2.0! 2.0 is much faster then >1.3.4. Make sure you use 2.0b7 which is even faster then b5. You can install >1.3.4 and 2.0b7 at then same time, so you can run your benchmark with both >modules. I would like to see a comparison of the two, that is not made by >myself. Anyway if you start a new project, you should build on top of 2.0 >and not 1.3.4, because 2.0 is not only faster, but also has much more >possibilities. > >Tell me what trouble you have with compiling and we will find what's ging >wrong. I took some time and went back and got 2.0b7 installed and ran the test for it and here are the numbers again. EM2 : Embperl-2.0b7. I also re-ran the Mason test as it was not doing the right thing to included files. It was just printing the contents of the 3 files instead of executing them and displaying the contents. This made the numbers closer to Embperl. 025 050 075 100 150 200 300 400 - PTT: 059 117 172 207 220 208 199 200 PHP: 060 118 173 211 235 252 249 241 ASP: 039 042 051 ** *** *** *** *** MAS: 057 098 114 114 108 094 067 064 EM2: 057 095 101 095 099 089 056 030 EMB: 040 040 039 039 028 026 * *** SSP: 060 118 172 218 246 268 277 277 - MAX: 062 125 187 250 294 294 294 294 For me these numbers are great and I plan on moving to 2.0 as soon as I can for my personal stuff. This works for me because I am only getting about 1 *request* per second, but at work we need to be able to deal with on average 18750 *page views* per second (1.62 billion page views per day on average). Of course this is spread across multiple machines, but you also need to take into consideration the number of machines needed, cost of machines, cost to maintain the software on all those machines, etc. Gerald I just want to say that I think you have created a GREAT thing with Embperl and I will always continue to use it. The whole point of the message was not to slam Embperl, but making sure I wasn't missing something in the configuration with mod_perl or Embperl. Thanks, Brian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance numbers, HELP!
What unit are your performance numbers? I was wondering what these numbers reflect and how they are measured. Thanks, Marcus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance numbers, HELP!
Hi Marcus, The top label numbers are the concurrent connections test and the data number are how many request per second the machine could output. All the test were done with Apache Benchmark. Brian At 8:19 PM -0400 4/19/02, Marcus Doemling wrote: >What unit are your performance numbers? I was wondering what >these numbers reflect and how they are measured. > >Thanks, >Marcus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance numbers, HELP!
> > At 8:19 PM -0400 4/19/02, Marcus Doemling wrote: > >What unit are your performance numbers? I was wondering what > >these numbers reflect and how they are measured. > The top label numbers are the concurrent connections test and the > data number are how many request per second the machine could output. > All the test were done with Apache Benchmark. > > Brian Thanks Marcus, I was actually wondering the same thing! Brian, Perhaps this is an obvious note that you caught, but you mentioned turning on debugging. You didn't do this for the *actual* test, did you? I understand that would have a performace effect. (you probably didn't, but I just thought I would mention it) Another question -- what exactly did you have each of the scripts doing? I examined a few of those packages you benchmarked recently, and it seems they had very different strengths and weaknesses. They each developed to fill a slightly different niche. I imagine such a rigorous performance test would be biased on what exactly you asked them to do for it. Perhaps I am mistaken, and if so -- please enlighten me! ;-) One thing I found slightly disturbing, was the high ranking of PHP numbers to the perl solutions. I imagined that PHP might win, as it is a web based option *designed* to be a web scripting language from the ground up -- but I did not expect such a large difference in numbers. I also want to thank you Brian, such a test was very interesting and thought provoking. Cameron PS -- Richter: *I* still think Embperl is the best. ;-) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Performance numbers, HELP!
At 9:39 PM -0400 4/19/02, Cameron McBride wrote: >Perhaps this is an obvious note that you caught, but you mentioned >turning on debugging. You didn't do this for the *actual* test, did >you? I understand that would have a performace effect. (you >probably didn't, but I just thought I would mention it) Yes debug was off for the test, but I ran a few before the test with it on to make sure it was caching. >Another question -- what exactly did you have each of the >scripts doing? The test file would include/execute 3 other files (header, footer, common section). 2 calls were made into our C/C++ code. Perl module wrappers were written for the mod_perl test and php to C/C++ extensions were written for the PHP test. One call was a simple string input, mangled in C/C++ and then displayed. The other call created a multi-level hash in C/C++ then copied into the native language (perl/PHP), then displayed it. This call also had a 400ms delay which was to simulate back end server delay, this is were the lower Max limit comes in (25/.4=62.5). > I examined a few of those packages you benchmarked >recently, and it seems they had very different strengths and >weaknesses. They each developed to fill a slightly different niche. >I imagine such a rigorous performance test would be biased on what >exactly you asked them to do for it. > Perhaps I am mistaken, and if so >-- please enlighten me! ;-) This is true. Our test page was setup to simulate the most common things we do (calls to our C/C++ code and servers). >One thing I found slightly disturbing, was the high ranking of PHP >numbers to the perl solutions. I imagined that PHP might win, as it >is a web based option *designed* to be a web scripting language from the >ground up -- but I did not expect such a large difference in numbers. Trust me, I feel the same way, but I have run the test multiple times and get the same numbers. Just to make sure you know that the PHP setup was with PHP Accelerator (http://www.php-accelerator.co.uk/). Two numbers that were recorded during the tests were idle times and virtual active memory. I only have those numbers for the PPT, PHP and SSP test, but what they showed were interesting. Our current proprietary technology used far less memory than PHP or mod_perl/SSP. PHP was in the middle for memory usage and mod_perl/SSP used *a lot* of memory. I believe the low numbers for Embperl after 100-150 current connections is because it runs low on memory and start swapping. This can be fixed by adding more memory, but this test server is close to *thousands* of boxes were have in production. I have run two more test for PHP, one with no caching and one with the Zend Optimizer at optimization_level=15. The Zend Optimizer doesn't help one bit, look at the numbers. PPT: Proprietary template technology DSO N-PHP: PHP 4.1.2 DSO with no caching O-PHP: PHP 4.1.2 DSO with Zend Optimizer (level=15) A-PHP: PHP 4.1.2 DSO with PHP Accelerator ASP: mod_perl 1.26 DSO with Apache::ASP 2.31 MAS: mod_perl 1.26 DSO with HTML::Mason 1.04 EM2: mod_perl 1.26 DSO with Embperl 2.0b7 EMB: mod_perl 1.26 DSO with HTML::Embperl 1.3.4 SSP: mod_perl 1.26 DSO with SimpleASP 025 050 075 100 150 200 300 400 concurrent connections - PTT: 059 117 172 207 220 208 199 200 requests per second N-PHP: 059 098 110 110 110 110 109 107 requests per second O-PHP: 058 096 107 106 107 106 106 106 requests per second A-PHP: 060 118 173 211 235 252 249 241 requests per second ASP: 039 042 051 ** *** *** *** *** requests per second MAS: 057 098 114 114 108 094 067 064 requests per second EM2: 057 095 101 095 099 089 056 030 requests per second EMB: 040 040 039 039 028 026 * *** requests per second SSP: 060 118 172 218 246 268 277 277 requests per second - MAX: 062 125 187 250 294 294 294 294 Max concurrent connections >I also want to thank you Brian, such a test was very interesting and thought >provoking. Thanks! >Cameron > >PS -- Richter: *I* still think Embperl is the best. ;-) I second that!! I took one file and tried to write it in PHP and it took 52 lines of code to do what I did in 25 with Embperl, I am sticking with Embperl! Brian - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
