I did a comparative load test http://blog.gidley.co.uk/2009/05/tapestry-load-testing-round-up.html a while back and we do a lot of load testing for SeeSaw.com (mainly to compare versions). I did create a 'comparative' struts application for that evaluation (source code http://sites.google.com/a/gidley.co.uk/tapestryloadtest/Home/test-5)
We use Grinder as our load test tool - as it handles clustered load generating agents well and the Jython scripting makes it easy to share steps between 'journeys' in the application. You probably don't need to worry about clustering - as it isn't a core tapestry 'feature' (rather an environmental one), but even without that feature grinder is easy to work and powerful. If your goal is to test tapestry the first issue you will hit is any Data Access - in our apps the data access piece is 90% of the page time. In SeeSaw we 'solve' this by heavily caching data so most pages only access data from a Cache. I would suggest to get a test that focused on tapestry you load you data from in memory caches. That way you won't be measuring how fast Hibernate or your DB(etc) performs. Another big issue we hit is breaking/maintaining our load test scripts - as they effectively reverse engineer some tapestry behaviours (especially AJAX ones) they can get broken quite easily. Our solution is to run them single threaded as part of our CI builds. This usually catches obvious changes. Another thing to watch out for is your app server. Jetty is an excellent high load container - but depending on how it is configured it can 'throttle' throughput (this is excellent thing in production). This can skew your test as you end up measuring the throttling and not the application. The trick is to set the thread pools in Jetty far higher than you would in production so you can be sure you are not maximising them. I would advise trying to use controlled hardware for your test - slight environmental differences can totally adjust the results. EC2 and its like are popular for this - do Apache have anything similar? I can get you time on (but not direct access to) the ioko cloud (which is a VMWare VSphere cluster), which does let you nearly guarantee QOS. It may be a good idea to build the test as something that can be easily deployed as a Virtual machine as it would let you easily deploy it anywhere. I have previously used Suse Studio (http://susestudio.com/) to build a very small VM with a maven script on it that updates/runs my app. The VM's built there will run pretty much anywhere. To find issues with a load test I recommend YourKit. It gives you a ton of monitoring and via its 'probes' you can see right inside tapestry. It is free for open source projects. I would also recommend enabling JMX in your application and periodically writing the stats somewhere. You can do this simply with VisualVM or you can get more advanced and store them in a database and graph them - ioko use Cacti http://www.cacti.net/ and http://jmxmonitor.sourceforge.net/ (though it may be a bit more complex for what you want). Ben Gidley On Thu, Sep 2, 2010 at 11:07 PM, Thiago H. de Paula Figueiredo < [email protected]> wrote: > On Thu, 02 Sep 2010 18:30:49 -0300, Howard Lewis Ship <[email protected]> > wrote: > > One of my long term and unrealized goals for Tapestry is to have a >> legitimate performance testing lab. >> > > Cool! > > > I'd like the ability to run a "standard" application through its >> paces, collect statistics, and see how well each new Tapestry release >> compares to its predecessor in terms of general performance: response >> time, memory utilization, saturation and recovery. Ideally, it would >> be nice to create equivalent JSP/Spring MVC/Struts/Wicket >> implementations of the same app. >> > > I've done something a little similar, but in a smaller scope, for a > presentation comparing Struts 1, Struts 2, JSF and Tapestry 5. I implemented > a simple application, DAOs and simple business rules layer, and them > implemented its web interface using each of the frameworks. > > > I'm really concerned with ensuring that 5.2 out-performs 5.1, and uses >> less memory. The singleton pages approach should lower the memory >> utilization ... but I'm concerned that the new AOP stuff inside >> ClassTransformation (which has a tendency to create lots of little >> one-off classes) will overshadow the other improvements. >> > > Someone in the mailing list mentioned Parfait ( > http://code.google.com/p/parfait/wiki/IntroductionToParfait). It could be > helpful for that. > > -- > Thiago H. de Paula Figueiredo > Independent Java, Apache Tapestry 5 and Hibernate consultant, developer, > and instructor > Owner, Ars Machina Tecnologia da Informação Ltda. > http://www.arsmachina.com.br > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > -- Ben Gidley www.gidley.co.uk [email protected]
