Hi there- https://issues.apache.org/jira/browse/HBASE-4448
I attached a prototype of HBaseTestingUtilityFactory to this ticket. I'd like to get some feedback on the general idea (emphasis on "general idea") and then if the consensus is that it makes sense I will continue and then we can do a more formal review on the reviews thing. Doug On 9/19/11 11:55 PM, "Stack" <[email protected]> wrote: >On Mon, Sep 19, 2011 at 1:06 PM, Jesse Yates <[email protected]> >wrote >>> ... will spent (at least on my laptop) about 10.7 seconds setting up >>>the >>> cluster, and 7.3 seconds tearing down. Assuming that we aren't >>>running in >>> a separate JVM each test invocation, sharing the same instance of >>> HBaseTestingUtility would be an enormous benefit. >>> > >Seems like it would be worth investigating makeing this run > > >>> 1) Re-use JVMs between test runs (if we are already doing this, then >>>no >>> action) >>> >> >> This is already built into maven when we run tests normally. This leads >>to >> some interesting situations for messing around with class loading/global >> variables; if you don't reset the environment, it can cause other >> (essentially non-broken) tests to fail. Unless we are running things in >> forked mode . >> >> Forked mode could also cut down on the straight unit test time, but in >>my >> experience this has lead to difficulty in figuring out what crashed. For >> some reason maven doesn't realize which tests failed, just that they >>did, in >> its final output (though it still stdouts "FAILURE"). >> > >Yeah, we run forked mode. > > <configuration> > ><forkedProcessTimeoutInSeconds>900</forkedProcessTimeoutInSeconds> > <argLine>-enableassertions -Xmx1400m</argLine> > <redirectTestOutputToFile>true</redirectTestOutputToFile> > </configuration> > >I think the thinking was fork it because so much is going on and yeah, >for sure there'll be cleanup afterward if all goes awry. > >Lohit suggested running tests in parallel. I tried it but things >didn't seem to run faster in my playing around. See HBASE-4207. > > >>> 2) Re-use the same HBaseTestingUtility instance for all possible >>>tests. >>> >>> I think #2 isn't "that bad" if we had a factory method to get/return >>> HBaseTestingUtility instance and internally this factory could do a >>> ref-count and auto-detect of non-activity, then it could shutdown the >>> cluster behind the scenes. >>> >>> >> +1 on this idea. >> We should also look into adding more/less region servers whenever people >> request more. We could just leave it up for the entire span of the >> integration tests and then bring it down afterwards (to make it easier >>on >> ourselves. >> > >We do some of this now. TestFromClientSide and TestAdmin are an >aggregation of a bunch of tests that used to each spin up own >clusters. > >We should do more of this. I think what happens is that a bunch of >our tests do one test only and to run the test, they spin up and down >a cluster. Could put a few together -- those that make sense. > >St.Ack
