I think the problem we are trying to solve is, "how do we run a bunch of integration test maven projects"
I'm still a little fuzzy on what's going on with the testsuite. What I think is that in 2.1 we used the genesis maven-maven-plugin, and jason dillon worked on it some more and turned it into the codehaus shitty-maven-plugin which was used during 2.2 development. It's much more sophisticated than the maven-maven-plugin so given a choice between the two I'd say we should use the shitty-maven-plugin. I don't think the shitty-maven-plugin is being very actively maintained. I don't see much evidence that the shitty-maven-plugin is actually configured in 2.2 or trunk. Maybe I didn't look in the right place. There is a third choice that I don't think was available when 2.2 was started, the maven-invoker-plugin. This is definitely supported and I think we should try to use it. I wonder if we should divide the tests into two groups, those that test starting and stopping the server, and those that require a running server (and don't stop it). While I think that the start-stop tests probably have to be run in order, I'd think that all the other test projects could probably be run in parallel. (a maven 3 feature). thanks david jencks On May 11, 2010, at 9:49 PM, Forrest Xia wrote: > Thank you, David, for sharing your thoughts here. See my response below. > > On Wed, May 12, 2010 at 6:18 AM, David Jencks <david_jen...@yahoo.com> wrote: > > On May 11, 2010, at 2:40 AM, Forrest Xia wrote: > >> Hi, >> >> I spent some time on reenabling G 2.2 branch testsuite to work. After >> comparing 2.2 branch with 2.1 branch, I would propose to revert back to the >> 2.1 configs, that is using maven-maven-plugin to run tests in child >> processes. The benefit is we can use testsuite as a unit to run children >> test cases while just start/stop geronimo server instance only one time, >> this will save time and disk space for testsuite execution. > > Aren't there a bunch of command tests that check that the start/stop server > commands work? How will this work with the single start stop? > I might not make myself clear here. Here I say "testsuite as a unit to run > children test cases while just start/stop geronimo server instance only one > time" means that, for each testsuite, the server only needs to start/stop > once if using maven-maven-plugin to run tests in a children process. That is, > for the current set of testsuites, we only need to start/stop server nine > times, since we have nine testsuites. > > > I would much rather move 2.2 towards what we're doing in trunk than revert > back to what is in 2.1 I think that since the failsafe plugin exists and we > know about it we should use it. Using standard maven plugins is almost > always a better idea than trying to come up with something ourselves. > Yes, using failsafe plugin to run integration test is the right thing to do. > Attached the updated poms I replace surefire with failsafe to run integration > test. > > > The only objection I can think of to using failsafe is that the server will > be started and stopped too often. > Yes, as you pointed out, if we use the thing like gtrunk testsuite, it will > cause too many times start/stop of server instance. This not only cost too > much time, but also consume too much disk space. I did a calculation, the > gtrunk way will cost at least 4G disk space for each run of testsuite(because > it needs to expand a server assembly for each of test backets). If we ADD > using maven-maven-plugin, it will save three fourth of time and disk space. > > I wonder if we could get around this by having a geronimo-maven-plugin mojo > that starts a server if one is not already running. We'd put this in the > pre-integration-test phase of every it and put nothing in the > post-integration-test phase. I think that, being in a child process, the > server would get killed when maven exits. We could also turn off the server > in the testsuite pom post-integration-test phase. > I think we could get around this by using maven-maven-plugin :-) > > I'm rather mystified as to what happened with the 2.2 testsuite. I thought I > remembered it running when I released 2.2, although I always got failures on > my machine. It is certainly not running now. It looks like I updated a > comment to suggest running with -Pit,all-subprojects although this will not > start the server before executing the tests. > > So, I'm against moving back to the 2.1 way of running the testsuite unless we > really can't get anything else to work. > Maybe I should not say "revert back", actually I think we should keep using > maven-maven-plugin because of the benefit it brings as stated above. > > thanks > david jencks > > > Please help review the updated poms and let me know if it is OK for you. > > Forrest > <testsuite-2.2.1-SNAPSHOT.pom><commands-testsuite-2.2.1-SNAPSHOT.pom><deploy-2.2.1-SNAPSHOT.pom>
