Akash, Do you know a way to handle this in maven? I haven't been able to find any docs on it (and haven't seen it anywhere). That's not to say that it doesn't exist, given the state of maven documentation.
If not, we should consider writing a wrapper around 'maven test' and 'maven verify' that will do the test rerunning and long test verification that you are talking about. Added a new ticket (HBASE-4480<https://issues.apache.org/jira/browse/HBASE-4480>) on this issue. -Jesse Yates On Sat, Sep 24, 2011 at 12:11 PM, Akash Ashok <thehellma...@gmail.com>wrote: > Hi All, > Just a suggestion about the running tests. We should also enable > re-running or retrying the failed or skipped test cases just to make sure > the result wasn't just an erroneous one. > > Considering the discussion started by Ted in mail chain below this would > result in better outcome thus taking out the developers or committer's time > in re-running the test cases. > > http://search-hadoop.com/m/r2Ab624ES3e/v=threaded > > Cheers, > Akash A > > On Thu, Sep 22, 2011 at 4:29 AM, Jesse Yates <jesse.k.ya...@gmail.com > >wrote: > > > Hi all, > > > > Added https://issues.apache.org/jira/browse/HBASE-4454 to track just > > adding > > the failsafe plugin to the regular build process. This is a stopgap till > we > > do the module split (which is going to be painful), to help speed up > > people's builds. > > > > -Jesse Yates > > > > On Tue, Sep 20, 2011 at 7:22 AM, Doug Meil < > doug.m...@explorysmedical.com > > >wrote: > > > > > > > > Hi there- > > > > > > https://issues.apache.org/jira/browse/HBASE-4448 > > > > > > I attached a prototype of HBaseTestingUtilityFactory to this ticket. > I'd > > > like to get some feedback on the general idea (emphasis on "general > > idea") > > > and then if the consensus is that it makes sense I will continue and > then > > > we can do a more formal review on the reviews thing. > > > > > > Doug > > > > > > > > > > > > > > > On 9/19/11 11:55 PM, "Stack" <st...@duboce.net> wrote: > > > > > > >On Mon, Sep 19, 2011 at 1:06 PM, Jesse Yates <jesse.k.ya...@gmail.com > > > > > >wrote > > > >>> ... will spent (at least on my laptop) about 10.7 seconds setting > up > > > >>>the > > > >>> cluster, and 7.3 seconds tearing down. Assuming that we aren't > > > >>>running in > > > >>> a separate JVM each test invocation, sharing the same instance of > > > >>> HBaseTestingUtility would be an enormous benefit. > > > >>> > > > > > > > >Seems like it would be worth investigating makeing this run > > > > > > > > > > > >>> 1) Re-use JVMs between test runs (if we are already doing this, > then > > > >>>no > > > >>> action) > > > >>> > > > >> > > > >> This is already built into maven when we run tests normally. This > > leads > > > >>to > > > >> some interesting situations for messing around with class > > loading/global > > > >> variables; if you don't reset the environment, it can cause other > > > >> (essentially non-broken) tests to fail. Unless we are running > things > > in > > > >> forked mode . > > > >> > > > >> Forked mode could also cut down on the straight unit test time, but > in > > > >>my > > > >> experience this has lead to difficulty in figuring out what crashed. > > For > > > >> some reason maven doesn't realize which tests failed, just that they > > > >>did, in > > > >> its final output (though it still stdouts "FAILURE"). > > > >> > > > > > > > >Yeah, we run forked mode. > > > > > > > > <configuration> > > > > > > > ><forkedProcessTimeoutInSeconds>900</forkedProcessTimeoutInSeconds> > > > > <argLine>-enableassertions -Xmx1400m</argLine> > > > > <redirectTestOutputToFile>true</redirectTestOutputToFile> > > > > </configuration> > > > > > > > >I think the thinking was fork it because so much is going on and yeah, > > > >for sure there'll be cleanup afterward if all goes awry. > > > > > > > >Lohit suggested running tests in parallel. I tried it but things > > > >didn't seem to run faster in my playing around. See HBASE-4207. > > > > > > > > > > > >>> 2) Re-use the same HBaseTestingUtility instance for all possible > > > >>>tests. > > > >>> > > > >>> I think #2 isn't "that bad" if we had a factory method to > get/return > > > >>> HBaseTestingUtility instance and internally this factory could do a > > > >>> ref-count and auto-detect of non-activity, then it could shutdown > > the > > > >>> cluster behind the scenes. > > > >>> > > > >>> > > > >> +1 on this idea. > > > >> We should also look into adding more/less region servers whenever > > people > > > >> request more. We could just leave it up for the entire span of the > > > >> integration tests and then bring it down afterwards (to make it > easier > > > >>on > > > >> ourselves. > > > >> > > > > > > > >We do some of this now. TestFromClientSide and TestAdmin are an > > > >aggregation of a bunch of tests that used to each spin up own > > > >clusters. > > > > > > > >We should do more of this. I think what happens is that a bunch of > > > >our tests do one test only and to run the test, they spin up and down > > > >a cluster. Could put a few together -- those that make sense. > > > > > > > >St.Ack > > > > > > > > >
