For this type of testing, I've used JMeter (http://jakarta.apache.org/jmeter/) successfully in the past. Without a doubt, you need at least two machines, perhaps more, otherwise the results will be skewed as you quickly exhaust the clients even if they don't fully parse the responses. I wouldn't worry about creating a real-world app, a synthetic one that exercises key parts of the framework should give more meaningful results. Memory and cpu resource consumption should be measured on the server, throughput from the (master) client.
I have not used Performance Hudson plugin (http://wiki.hudson-ci.org/display/HUDSON/Performance+Plugin) but it seems ideal for this. I might be interested in participating in this effort. There are several things that need to be done - securing dedicated hardware is surprisingly one of the most difficult issues (anybody can find a few boxes for temporary use, but how do you make sure you can keep the environment stable and unchanged for years to come). A desktop or two running under somebody's desk is not a great solution in the long run. Kalle On Thu, Sep 2, 2010 at 2:30 PM, Howard Lewis Ship <[email protected]> wrote: > One of my long term and unrealized goals for Tapestry is to have a > legitimate performance testing lab. > > I'd like the ability to run a "standard" application through its > paces, collect statistics, and see how well each new Tapestry release > compares to its predecessor in terms of general performance: response > time, memory utilization, saturation and recovery. Ideally, it would > be nice to create equivalent JSP/Spring MVC/Struts/Wicket > implementations of the same app. > > I'm really concerned with ensuring that 5.2 out-performs 5.1, and uses > less memory. The singleton pages approach should lower the memory > utilization ... but I'm concerned that the new AOP stuff inside > ClassTransformation (which has a tendency to create lots of little > one-off classes) will overshadow the other improvements. > > So ... how do we go from thinking about this to doing it? I have a > spare desktop I could use for this purpose (I'm going to wipe out > windows and install Ubuntu). It's dated hardware but otherwise > probably ok. Does any of the committers, or Tapestry community, have > some specific skills, or better ideas? > > > -- > Howard M. Lewis Ship > > Creator of Apache Tapestry > > The source for Tapestry training, mentoring and support. Contact me to > learn how I can get you up and productive in Tapestry fast! > > (971) 678-5210 > http://howardlewisship.com > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
