-- dandans <[EMAIL PROTECTED]> wrote (on Saturday, 18 October 2008, 07:56 PM -0700): > holografix wrote: > > If you download benchmark scripts from > > http://www.yiiframework.com/performance/ you will see that zf benchmark is > > not correct. > > From readme.txt: > > Set the PHP include path by appending the path to the Zend framework > > - Overwrite > > "/demos/Zend/Wildfire/application/controllers/IndexController.php" with > > the > > one in this folder. > > - Run with "/demos/Zend/Wildfire/public/index.php" > > > > To do a simple "Hello world" you don't need Registry, Log, DB components. > > > > Don't trust in that benchmark results. > > I'm not quite sure why Zend with APC is slower, but that is the truth. I > repeated the test many times and got the same result. So would you please > create a ZF application that you think is fair for that comparison? I'm > willing to update it and re-do the test. > > As described, the purpose of the test is to reveal the upper limit of each > framework, or the minimal overhead that each framework incurs. By no means > it attempts to claim you should choose a framework purely based on this > performance comparison.
Okay, I'm finally going to break my self-imposed silence on this issue. First off, web applications are rarely deployed on Windows, and when deployed on Windows, rarely on Apache (IIS with FastCGI is the best deployment option for PHP with Windows). The majority of hosting environments will be running a *nix-based OS, with either apache or lighttpd. So, this is the first criticism I have of the benchmark. Second, you provide no details about how APC is actually configured for your benchmarks. Considering there have been a number of other benchmarks publically published that all show substantial improvement of framework performance (and not just ZF) with APC, I have to question if you have properly configured APC. Third, while I understand the idea behind these "upper limit" benchmarks, I also feel they advocate poor application and systems architecture. You should be caching whenever possible, and in caching, you will get much, much better performance with your framework-based applications than you can get with these "upper limit" tests: the best caching strategies will bypass the PHP engine entirely; next-best will bypass your framework. Fourth, in looking at the "application" benchmarked, the ZF version is incomplete at best, and completely flawed at worst. It does not follow any sort of established guidelines or conventions, in our own quick start guide or otherwise, and clearly doesn't contain even the full minimal files necessary to run it. Calling "echo" and "die" from an action method (as it does) is ill-advised, and also means that you're loading a ton of functionality that you're never using (ViewRenderer, ErrorHandler, response object, etc) -- giving skewed results. Fifth, in our own benchmarking and profiling that we are performing for the 1.7.0 release, we're doing more than a single "Hello World" page. We're testing a logical progression of pages, and performing this progression over a period of time with increasing amounts of traffic. This allows us to see what the true upper limit of the application is, and also identify areas where caching strategies can make significant performance improvements. When a framework makes effective use of class lazy-loading (as we recommend and principally adhere to), caching can significantly reduce the resources used per-request; if pages can be pre-generated, you can even reduce the overhead of the PHP engine to none. -- Matthew Weier O'Phinney Software Architect | [EMAIL PROTECTED] Zend Framework | http://framework.zend.com/
