See my comments below. On Thu, Nov 19, 2009 at 9:12 AM, Markus Kohler <[email protected]> wrote: > Hi all, > See my comments below... > > On Thu, Nov 19, 2009 at 8:27 AM, Richard Hirsch <[email protected]>wrote: > >> I enhanced the performance test page in the wiki >> (http://cwiki.apache.org/confluence/display/ESME/Performance+tests) >> with more details and test ideas >> >> > I like that. > > >> @Markus: I think it would great if you could do the performance tests >> once a week on the Stax performance instance. Initially, it would be >> great just to have test suite that we can run. It is critical that we >> can put such tests into our continuous integration cycle but until >> this is possible, it would be great if you could just do these short >> tests on a weekly basis. That would allow us to make sure that code >> changes aren't impacting our performance. I'd like to make these tests >> as easy for you to perform as possible. >> >> > The tests could also be used as simple functional tests. They would have > catched for example the javascript problem that we had lately. > > >> Although regression testing would be one goal, the changing of small >> parameters (adding one action for each user) and seeing the impact on >> performance would also be very interesting. >> >> Yes. I certainly need some simple extensions to what I've done so far. For > example users should follow other users. > I think this way we could even manually do some simple performance > tests/analysis. > > >> We should probably define some sort of a process that describes the >> tests to make sure that the basis is always the same. For example, >> >> 1. Drop existing DB - this can be via a stax command >> > > Yes for the regression test that is a must. > > >> 2. Create existing DB - this can be via a stax command >> > Not sure, what you mean. Can we switch from one DB to another, for example > with different data sets (nr. of Users)? > That would be fantastic.
You can only do this at deplyoment time. By changing some stax configuration files. No problem but only at deployment and not-run-time > > >> 3. Deploy application - - this can be via a stax command >> 4. Test - this could be done via maven >> 5. Take screenshots of Stax console (still have to define which >> screens we need - maybe like the ones you sent me last time) >> > > Stax doesn't have a way to get the data as a file (CSV or whatever)? You can access the mysql database via jdbc - there are details on the stax console page. This is probably the best way to do it. I didn't have access to these ports, because of corporate firewall restrictions. > I could use Selenium to navigate to the Stax console at the end of the test > and do some screenshots automatically. That would be very cool. > > >> 6. Mail all the details to me and I post to wiki >> >> > Yes. I can do that. > > My plan is to get the low hanging fruits first of course. > Those are (at the top of my head): > - Checking the client side. E.g. cache settings, request compression etc. . > Not sure whether I should do it now, because we will get a new UI hopefully > soon? > - memory usage analysis. This is low hanging, because I've done it so often, > I could do it while sleeping ;) > - quick memory allocation tracing analysis. E.g. how much memory is > allocated and reclaimed for one interaction step. Can be done manually, btw > does someone know how to tell maven "jetty:run" which JVM to use? > Added these ideas to the wiki page > Regards, > Markus > > >> Thoughts? >> >> D. >> >
