Just blogged about performance tests: http://blog.esme.us/return-of-the-performance-test/
D. On Thu, Nov 19, 2009 at 9:22 AM, Richard Hirsch <[email protected]> wrote: > See my comments below. > > On Thu, Nov 19, 2009 at 9:12 AM, Markus Kohler <[email protected]> > wrote: >> Hi all, >> See my comments below... >> >> On Thu, Nov 19, 2009 at 8:27 AM, Richard Hirsch <[email protected]>wrote: >> >>> I enhanced the performance test page in the wiki >>> (http://cwiki.apache.org/confluence/display/ESME/Performance+tests) >>> with more details and test ideas >>> >>> >> I like that. >> >> >>> @Markus: I think it would great if you could do the performance tests >>> once a week on the Stax performance instance. Initially, it would be >>> great just to have test suite that we can run. It is critical that we >>> can put such tests into our continuous integration cycle but until >>> this is possible, it would be great if you could just do these short >>> tests on a weekly basis. That would allow us to make sure that code >>> changes aren't impacting our performance. I'd like to make these tests >>> as easy for you to perform as possible. >>> >>> >> The tests could also be used as simple functional tests. They would have >> catched for example the javascript problem that we had lately. >> >> >>> Although regression testing would be one goal, the changing of small >>> parameters (adding one action for each user) and seeing the impact on >>> performance would also be very interesting. >>> >>> Yes. I certainly need some simple extensions to what I've done so far. For >> example users should follow other users. >> I think this way we could even manually do some simple performance >> tests/analysis. >> >> >>> We should probably define some sort of a process that describes the >>> tests to make sure that the basis is always the same. For example, >>> >>> 1. Drop existing DB - this can be via a stax command >>> >> >> Yes for the regression test that is a must. >> >> >>> 2. Create existing DB - this can be via a stax command >>> >> Not sure, what you mean. Can we switch from one DB to another, for example >> with different data sets (nr. of Users)? >> That would be fantastic. > > You can only do this at deplyoment time. By changing some stax > configuration files. No problem but only at deployment and > not-run-time > >> >> >>> 3. Deploy application - - this can be via a stax command >>> 4. Test - this could be done via maven >>> 5. Take screenshots of Stax console (still have to define which >>> screens we need - maybe like the ones you sent me last time) >>> >> >> Stax doesn't have a way to get the data as a file (CSV or whatever)? > > > You can access the mysql database via jdbc - there are details on the > stax console page. This is probably the best way to do it. I didn't > have access to these ports, because of corporate firewall > restrictions. > >> I could use Selenium to navigate to the Stax console at the end of the test >> and do some screenshots automatically. > > That would be very cool. > >> >> >>> 6. Mail all the details to me and I post to wiki >>> >>> >> Yes. I can do that. >> >> My plan is to get the low hanging fruits first of course. >> Those are (at the top of my head): >> - Checking the client side. E.g. cache settings, request compression etc. . >> Not sure whether I should do it now, because we will get a new UI hopefully >> soon? >> - memory usage analysis. This is low hanging, because I've done it so often, >> I could do it while sleeping ;) >> - quick memory allocation tracing analysis. E.g. how much memory is >> allocated and reclaimed for one interaction step. Can be done manually, btw >> does someone know how to tell maven "jetty:run" which JVM to use? >> > > Added these ideas to the wiki page > >> Regards, >> Markus >> >> >>> Thoughts? >>> >>> D. >>> >> >
