I agree with Nicolas that this article deserves serious consideration by Fabien et al, especially since I don't agree that it's "Yet Another Troll/FUD" article. Given how good this article is in most respects, I expect it to be cited often.
The article is a revised (after responding to criticism of the first tests) benchmark of Cake, Solar, Symfony, and Zend. The author is the developer of Solar and is apparently performing the benchmarks mostly to see how Solar compares to the other 3 frameworks. He chose these because... "In my opinion, the frameworks in this comparison are the most full-featured
of those available for PHP. They make good use of design patterns, provide front and page controllers, allow for controller/view separation, and offer an existing set of plugins or added functionality like caching, logging, access control, authentication, form processing, and so on."
Symfony is included because he considers it one of the best, not because it's the most vulnerable to criticism. He's quite clear that these benchmarks are intended to test the overall responsiveness of the underlying framework, stripped of database access. From a response to a comment...
"...If your application is based on Framework X, and it's not running as fast as you'd like, it's good to know the theoretical top limit, so you can decide if that top limit will be enough. That way you're not trying get more from the code than is imposed by the framework itself. If it's not enough, it's time to choose a different component (server, language, framework, or something else) — or it's time to start scaling in some fashion."
Aside from the fact that by design a test like this is not at all real-world, his methodology is excellent, he's revealed the code, he's clear about his involvement with Solar (and Zend), and is up front about his motives. He appears to be testing Symfony without propel (no database access) and is testing both 0.6.3 and 1.0.0beta2 (he notes the speed improvement). But there may be problems with the way he has evaluated Symfony in particular. Again from the comments...
"Niko: BTW, did you deactivate web debug toolbar and dev level logs in your Symfony test app ? It consumes *huge* amount of processing time… pmjones: I bet there are several ways that each framework could be optimized to perform these tests better (such as your symfony suggestions) but that's not the idea. It wouldn't be fair to start "disabling" things on Symfony to make it run faster but leave the other frameworks as-is."
It's clear that Mr. Jones could use a bit of help with his Symfony tests, and it would be an excellent opportunity for the developers to address the issue of benchmarks in general (hopefully without getting too defensive), since one of the frequent questions about Symfony has been performance and, despite the fact that performance has significantly improved, there are good and valid reasons why Symfony doesn't appear to perform well compared to other frameworks in raw speed tests like this. In the conclusion to his article he issues an invitation... "Again, it is entirely possible that I have failed to incorporate important optimizations to one or more of the frameworks tested. If so, it is through ignorance and not maliciousness. If you are one of the lead developers on the frameworks benchmarked here and feel your project is unfairly represented, please contact me personally, and I'll forward a tarball of the entire benchmark system (including the framework code as tested) so you can see where I may have gone wrong." It seems to me that one of the Symfony leads should take him up on that offer, especially since his response will reveal much about his objectivity and fairness. Jon Phipps On 1/2/07, Nicolas Perriault <[EMAIL PROTECTED]> wrote:
http://paul-m-jones.com/blog/?p=238 Yet Another Troll/FUD on Symfony perfs, but I think someone of the staff could answer to this (and ask for the benches apps sources too.) ++ -- Nicolas Perriault http://www.clever-age.com Clever Age - Conseil en architecture technique GSM: +33 6 60 92 08 67 Tél: +33 1 53 34 66 10 >
-- Jon --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "symfony developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/symfony-devs?hl=en -~----------~----~----~----~------~----~------~--~---
