Matthew Weier O'Phinney-3 wrote:
>
> -- dandans <[EMAIL PROTECTED]> wrote
> (on Saturday, 18 October 2008, 07:56 PM -0700):
>> holografix wrote:
>> > If you download benchmark scripts from
>> > http://www.yiiframework.com/performance/ you will see that zf benchmark
>> is
>> > not correct.
>> > From readme.txt:
>> > Set the PHP include path by appending the path to the Zend framework
>> > - Overwrite
>> > "/demos/Zend/Wildfire/application/controllers/IndexController.php" with
>> > the
>> > one in this folder.
>> > - Run with "/demos/Zend/Wildfire/public/index.php"
>> >
>> > To do a simple "Hello world" you don't need Registry, Log, DB
>> components.
>> >
>> > Don't trust in that benchmark results.
>>
>> I'm not quite sure why Zend with APC is slower, but that is the truth. I
>> repeated the test many times and got the same result. So would you please
>> create a ZF application that you think is fair for that comparison? I'm
>> willing to update it and re-do the test.
>>
>> As described, the purpose of the test is to reveal the upper limit of
>> each
>> framework, or the minimal overhead that each framework incurs. By no
>> means
>> it attempts to claim you should choose a framework purely based on this
>> performance comparison.
>
> Okay, I'm finally going to break my self-imposed silence on this issue.
>
> First off, web applications are rarely deployed on Windows, and when
> deployed on Windows, rarely on Apache (IIS with FastCGI is the best
> deployment option for PHP with Windows). The majority of hosting
> environments will be running a *nix-based OS, with either apache or
> lighttpd. So, this is the first criticism I have of the benchmark.
>
> Second, you provide no details about how APC is actually configured for
> your benchmarks. Considering there have been a number of other
> benchmarks publically published that all show substantial improvement of
> framework performance (and not just ZF) with APC, I have to question if
> you have properly configured APC.
>
> Third, while I understand the idea behind these "upper limit"
> benchmarks, I also feel they advocate poor application and systems
> architecture. You should be caching whenever possible, and in caching,
> you will get much, much better performance with your framework-based
> applications than you can get with these "upper limit" tests: the best
> caching strategies will bypass the PHP engine entirely; next-best will
> bypass your framework.
>
> Fourth, in looking at the "application" benchmarked, the ZF version is
> incomplete at best, and completely flawed at worst. It does not follow
> any sort of established guidelines or conventions, in our own quick
> start guide or otherwise, and clearly doesn't contain even the full
> minimal files necessary to run it. Calling "echo" and "die" from an
> action method (as it does) is ill-advised, and also means that you're
> loading a ton of functionality that you're never using (ViewRenderer,
> ErrorHandler, response object, etc) -- giving skewed results.
>
> Fifth, in our own benchmarking and profiling that we are performing for
> the 1.7.0 release, we're doing more than a single "Hello World" page.
> We're testing a logical progression of pages, and performing this
> progression over a period of time with increasing amounts of traffic.
> This allows us to see what the true upper limit of the application is,
> and also identify areas where caching strategies can make significant
> performance improvements. When a framework makes effective use of class
> lazy-loading (as we recommend and principally adhere to), caching can
> significantly reduce the resources used per-request; if pages can be
> pre-generated, you can even reduce the overhead of the PHP engine to
> none.
>
> --
> Matthew Weier O'Phinney
> Software Architect | [EMAIL PROTECTED]
> Zend Framework | http://framework.zend.com/
>
>
First of all, I want to disclaim that the performance test was not intended
to attack ZF. In fact, the very original version didn't even consist of ZF
because I couldn't find a suitable test application.
I agree with you that the testing environment may be flawed, and I believe
ZF will perform much better on Linux/Unix. I will re-design the tests and
re-do it to make it more rigorous.
I do not agree with your third and fifth points, however.
While unarguable that server caching techniques can significantly improve
the performance of a Web application, we do not talk about them here because
it is simply off the point. For example, using Squid, I can get 10K RPS with
each framework, then what's the meaning of all these tests?
The so-called "upper limit" tests are designed to be the ones that can
reveal the minimal overhead of each framework without the help of server
caching techniques. Keep it in mind that these upper limits are different
from the numbers that real world applications can reach. So it doesn't make
sense if you try to claim your testing applications are more reasonable
simply because they are more realistic. You just couldn't compare them
because the goals are different.
--
View this message in context:
http://www.nabble.com/Framework-speed-shotout----question-tp19914787p20060128.html
Sent from the Zend Framework mailing list archive at Nabble.com.
