On 5/11/09, Alin Dreghiciu <[email protected]> wrote: > Lately I saw that the "biggest" complain about pax Exam is about the fact > that it takes a long time to run the tests. So, the main scope of this > thread is just to gather ideas on how we can speed it up.The parts where Pax > Exam/Pax Runner spends time I can think of are: > > 1. Scanning. Depending on what type of scanner and on what type of url the > scanner is working the scanning process can be slow. > > 2. Resolving maven artifacts without an exact version. If you are using the > mvn: url handler without a version the handler will have to go to each > available repository and determine what versions are available in that > repository in order to determine the highest version = the version to be > used. This can be a slow process. > > 3. Resolving profiles without a version. Profiles are transformed to > composite scanner + maven resolver. So, in case that no version is specified > same reasoning applies as above. > > 4. Download bundles to be provisioned (result of scanning) into Pax Runner > cache folder. Based on source and size can be slow. > > 5. Every test is run in its own instance of Pax Runner in order to avoid > side effects. > > 6. Pax Runner starts a new Process for each test. When launching the target > platform Pax Runner will start it in a new process. > > > For some time now I'm thinking on doing some parts in parallel in Pax > Runner, but not yet time to implement it. This will mean that points 1,2,3,4 > from above may take less time as they are done in parallel not sequential as > currently. The downside (but I can live with it without any problems :) is > that now download feedback will be shown. > > For item 2 and 3 the simplest solution is to specify an exact version. > Beside an exact version another improvement is to have the maven artifacts > in your local maven repository. So, if you are using maven for build it > would be easy to add the necessary artifacts to your dependencies. Same goes > for profiles and you can find an example in Apache Felix Karaf on how to do > that. > > Regarding item 6 Toni is working on an test container for felix > PAXEXAM-79<http://issues.ops4j.org/browse/PAXEXAM-79>. > I also just added a new feature to Pax Runner that will start the target > framework in the same process > PAXRUNNER-250<http://issues.ops4j.org/browse/PAXRUNNER-250>. > I used the new in process executor of Pax Runner as a POC (does not play > very well with Pax Exam) and the improvements I got was around 300 ms > (empirical observation of logged times). So, I would say that starting the > target framework in the the same process will not be a very big improvement. > And if compared the improvements to the side effects of starting more > frameworks many times in the same process, I would not go for it. To not > talk about the fact that supporting versions beside the latest is not a > trivial job. > > The biggest improvement I see is about not starting the container for each > test but start them for groups of tests that share the same configuration. > What I'm thinking of is to change the default behavior that starts a test > per test method, to first determine which tests (in the same test class) > share the same configuration and then start the test container per > configuration and run the tests. Your mileage can vary but if for example > you have 10 tests and one configuration the time it takes to run the tests > can dramatically go down. Not to talk about 100 tests ;) > More, we should allow the user to override this behavior by means of an > annotation on method/class level that specifies that the tests should be run > in their own instance.
I guess thats a very good idea. Still, i would implement those things as separate execution strategies so one can sacrifice speed for side-effect tradeoffs. Another (probably interesting) strategy is to launch the tests, as they do currently, in parallel. So they will get their own rmi port, paxrunner cache etc. Of cause this really scales when sharing downloaded artifacts at least. > > WDYT? > > -- > Alin Dreghiciu > Software Developer - Looking for new projects! > My profile: http://www.linkedin.com/in/alindreghiciu > My blog: http://adreghiciu.blogspot.com > http://www.ops4j.org - New Energy for OSS Communities - Open Participation > Software. > http://www.qi4j.org - New Energy for Java - Domain Driven Development. > -- Toni Menzel Independent Software Developer - Looking for new projects! Professional Profile: http://www.osgify.com Blog: tonitcom.blogspot.com [email protected] http://www.ops4j.org - New Energy for OSS Communities - Open Participation Software. _______________________________________________ general mailing list [email protected] http://lists.ops4j.org/mailman/listinfo/general
