Hi Gary, The hbase specific versions for surefire & junit are ready (see HBASE-4763 for all the details). Would it be possible to use your repository? What would you need from me? I've never deployed anything in a maven repo. Surefire works with maven, so it should be easy, JUnit does not, so there could be more work to do.
I copy the dev list for information. Regards, N. ---------- Forwarded message ---------- From: Gary Helmling <[email protected]> Date: Thu, Oct 20, 2011 at 11:50 PM Subject: Re: [unit tests] small tests vs. others To: [email protected] +1 on small/medium/large I have a test mvn repo up on people.apache.org that we can use for a patched surefire build, if that helps. On Thu, Oct 20, 2011 at 2:33 PM, Jonathan Gray <[email protected]> wrote: > +1 on small/medium/large. > >> -----Original Message----- >> From: Todd Lipcon [mailto:[email protected]] >> Sent: Thursday, October 20, 2011 1:49 PM >> To: [email protected] >> Subject: Re: [unit tests] small tests vs. others >> >> +1 on separating into small/medium/large. One of the requirements of >> "small" should be that it can run parallel with other tests and in nofork mode. >> >> See SUREFIRE-329 for the necessary surefire patch. Until that's integrated >> upstream we'd have to have our own patched surefire :( >> >> -Todd >> >> On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <[email protected]> wrote: >> > Hi, >> > >> > I've done a stat on the tests we have. Many of them are actually quite >> > simple: There are 226 test classes in HBase. 100 of them does not >> > use a cluster, and takes less than 2 seconds to execute. 24 do not use >> > a cluster but need between 2 and 40 seconds. >> > >> > I tried to execute these with different maven settings. >> > fork: once or always, i.e. all the tests are executed in the same JVM >> > or there is a new JVM created for every test class parallel; >> > deactivated or activated, with 3 threads and a per class >> > parallelisation. >> > >> > The 100 tests executes for the whole maven stuff, including the >> > initial dependency & task analysis fork always: 1m50s fork once: 30 >> > seconds >> > parallel: 30s (=> no difference). >> > >> > It works quite well, except with TestLruBlockCache that I had to remove. >> > >> > I tried as well with the 24 'bigger' ones but no cluster, for the >> > following >> > results: >> > fork always: 7m36s >> > parallel: 2m32s: 3 times faster. >> > >> > However, I had to remove many of them, there are some conflicts that I >> > have not solved (nor tried to). >> > >> > The fact that nearly 50% of the tests can be executed in 1 minutes is >> > interesting imho. >> > >> > That makes me wonder if we should try to get a separation of the tests >> > in 3 parts, small/medium/large as of >> > http://googletesting.blogspot.com/2009/07/software-testing-categorizat >> > ion.html >> > >> > small would be: >> > - less than two seconds, can be executed in the same jvm >> > - all of them: less than 2 minute >> > - no cluster, nothing that can break the environment >> > - run by the developers as a part of the build >> > >> > medium: >> > - less than 40 seconds >> > - all of them: would last less than 30 minutes, ideally 15 minutes >> > - run in an isolated jvm >> > - run explicitly by the developers before submitting a fix >> > >> > large: >> > - run in an isolated jvm >> > - Should last less than 45minutes >> > - run on the central built or explicitly by the developers >> > - break the built when it fails. >> > >> > Only one issue: it's not directly possible with maven, but there are >> > known workarounds :-) >> > >> > >> > What do you think? >> > >> > >> > Below the tests set for the 120 tests that seems to work altogether. >> > >> > >> > mvn test >> > - >> Dtest=org.apache.hadoop.hbase.avro.TestAvroUtil,org.apache.hadoop.hba >> > se.client.TestAttributes,org.apache.hadoop.hbase.client.TestGet,org.ap >> > ache.hadoop.hbase.client.TestOperation,org.apache.hadoop.hbase.client. >> > TestResult,org.apache.hadoop.hbase.client.TestScan,org.apache.hadoop.h >> > >> base.coprocessor.TestRegionObserverStacking,org.apache.hadoop.hbase.fi >> > lter.TestBitComparator,org.apache.hadoop.hbase.filter.TestColumnPagina >> > tionFilter,org.apache.hadoop.hbase.filter.TestDependentColumnFilter,or >> > g.apache.hadoop.hbase.filter.TestFilterList,org.apache.hadoop.hbase.fi >> > lter.TestInclusiveStopFilter,org.apache.hadoop.hbase.filter.TestPageFi >> > lter,org.apache.hadoop.hbase.filter.TestParseFilter,org.apache.hadoop. >> > hbase.filter.TestPrefixFilter,org.apache.hadoop.hbase.filter.TestRando >> > >> mRowFilter,org.apache.hadoop.hbase.filter.TestSingleColumnValueExclude >> > Filter,org.apache.hadoop.hbase.filter.TestSingleColumnValueFilter,org. >> > apache.hadoop.hbase.io.hfile.slab.TestSlabCache,org.apache.hadoop.hbas >> > e.io.hfile.slab.TestSlab,org.apache.hadoop.hbase.io.hfile.TestBlockCac >> > >> heColumnFamilySummary,org.apache.hadoop.hbase.io.hfile.TestCachedBloc >> k >> > Queue,org.apache.hadoop.hbase.io.hfile.TestFixedFileTrailer,org.apache >> > .hadoop.hbase.io.hfile.TestHFileReaderV1,org.apache.hadoop.hbase.io.hf >> > ile.TestHFile,org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2,org.a >> > >> pache.hadoop.hbase.io.hfile.TestReseekTo,org.apache.hadoop.hbase.io.hf >> > ile.TestSeekTo,org.apache.hadoop.hbase.io.TestHalfStoreFileReader,org. >> > >> apache.hadoop.hbase.io.TestHbaseObjectWritable,org.apache.hadoop.hbas >> e >> > >> .io.TestHeapSize,org.apache.hadoop.hbase.io.TestImmutableBytesWritable >> > >> ,org.apache.hadoop.hbase.mapreduce.TestSimpleTotalOrderPartitioner,org >> > >> .apache.hadoop.hbase.mapreduce.TestTableSplit,org.apache.hadoop.hbase >> . >> > >> master.TestClockSkewDetection,org.apache.hadoop.hbase.master.TestDea >> dS >> > >> erver,org.apache.hadoop.hbase.master.TestDefaultLoadBalancer,org.apach >> > >> e.hadoop.hbase.master.TestMasterStatusServlet,org.apache.hadoop.hbase. >> > >> metrics.TestMetricsMBeanBase,org.apache.hadoop.hbase.monitoring.Test >> Me >> > >> moryBoundedLogMessageBuffer,org.apache.hadoop.hbase.monitoring.Test >> Tas >> > >> kMonitor,org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionH >> > >> andler,org.apache.hadoop.hbase.regionserver.TestCompactSelection,org.a >> > pache.hadoop.hbase.regionserver.TestExplicitColumnTracker,org.apache.h >> > >> adoop.hbase.regionserver.TestHRegionInfo,org.apache.hadoop.hbase.regio >> > >> nserver.TestKeyValueHeap,org.apache.hadoop.hbase.regionserver.TestKey >> V >> > >> alueScanFixture,org.apache.hadoop.hbase.regionserver.TestKeyValueSkipL >> > >> istSet,org.apache.hadoop.hbase.regionserver.TestMemStoreLAB,org.apach >> e >> > >> .hadoop.hbase.regionserver.TestMinVersions,org.apache.hadoop.hbase.reg >> > >> ionserver.TestQueryMatcher,org.apache.hadoop.hbase.regionserver.TestRe >> > gionSplitPolicy,org.apache.hadoop.hbase.regionserver.TestResettingCoun >> > ters,org.apache.hadoop.hbase.regionserver.TestRSStatusServlet,org.apac >> > >> he.hadoop.hbase.regionserver.TestScanDeleteTracker,org.apache.hadoop.h >> > >> base.regionserver.TestScanWildcardColumnTracker,org.apache.hadoop.hbas >> > >> e.regionserver.TestScanWithBloomError,org.apache.hadoop.hbase.regionse >> > >> rver.TestStoreScanner,org.apache.hadoop.hbase.regionserver.wal.TestHLo >> > >> gBench,org.apache.hadoop.hbase.regionserver.wal.TestHLogMethods,org.a >> p >> > >> ache.hadoop.hbase.regionserver.wal.TestWALActionsListener,org.apache.h >> > >> adoop.hbase.rest.model.TestCellModel,org.apache.hadoop.hbase.rest.mod >> e >> > >> l.TestCellSetModel,org.apache.hadoop.hbase.rest.model.TestColumnSchem >> a >> > >> Model,org.apache.hadoop.hbase.rest.model.TestRowModelo,rg.apache.ha >> doo >> > >> p.hbase.rest.model.TestScannerModel,org.apache.hadoop.hbase.rest.mod >> el >> > >> .TestStorageClusterStatusModel,org.apache.hadoop.hbase.rest.model.Test >> > >> StorageClusterVersionModel,org.apache.hadoop.hbase.rest.model.TestTabl >> > >> eInfoModel,org.apache.hadoop.hbase.rest.model.TestTableListModel,org.a >> > >> pache.hadoop.hbase.rest.model.TestTableRegionModel,org.apache.hadoop >> .h >> > >> base.rest.model.TestTableSchemaModel,org.apache.hadoop.hbase.rest.mo >> de >> > l.TestVersionModel,org.apache.hadoop.hbase.security.TestUser,org.apach >> > >> e.hadoop.hbase.TestAcidGuarantees,org.apache.hadoop.hbase.TestCompar >> e, >> > >> org.apache.hadoop.hbase.TestHRegionLocation,org.apache.hadoop.hbase.T >> e >> > >> stHServerAddress,org.apache.hadoop.hbase.TestHServerInfo,org.apache.ha >> > doop.hbase.TestKeyValue,org.apache.hadoop.hbase.TestSerialization,org. >> > >> apache.hadoop.hbase.TestServerName,org.apache.hadoop.hbase.util.TestB >> a >> > se64,org.apache.hadoop.hbase.util.TestByteBloomFilter,org.apache.hadoo >> > p.hbase.util.TestBytes,org.apache.hadoop.hbase.util.TestCompressionTes >> > t,org.apache.hadoop.hbase.util.TestFSTableDescriptors,org.apache.hadoo >> > p.hbase.util.TestHBaseFsckComparator,org.apache.hadoop.hbase.util.Test >> > >> IncrementingEnvironmentEdge,org.apache.hadoop.hbase.util.TestKeying,or >> > >> g.apache.hadoop.hbase.util.TestPoolMap,org.apache.hadoop.hbase.util.Te >> > >> stRegionSplitCalculator,org.apache.hadoop.hbase.zookeeper.TestHQuorum >> P >> > >> eer,org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServerArg,org >> .a >> > >> pache.hadoop.hbase.coprocessor.TestCoprocessorInterface,org.apache.had >> > oop.hbase.executor.TestExecutorService,org.apache.hadoop.hbase.filter. >> > TestColumnPrefixFilter,org.apache.hadoop.hbase.filter.TestFilter,org.a >> > pache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter,org.apache.ha >> > doop.hbase.io.hfile.slab.TestSingleSizeCache,org.apache.hadoop.hbase.i >> > o.hfile.TestCacheOnWrite,org.apache.hadoop.hbase.io.hfile.TestHFileBlo >> > ckIndex,org.apache.hadoop.hbase.io.hfile.TestHFileBlock,org.apache.had >> > oop.hbase.io.hfile.TestHFilePerformance,org.apache.hadoop.hbase.io.hfi >> > >> le.TestHFileSeek,org.apache.hadoop.hbase.ipc.TestDelayedRpc,org.apache >> > >> .hadoop.hbase.master.TestCatalogJanitor,org.apache.hadoop.hbase.region >> > >> server.TestAtomicOperation,org.apache.hadoop.hbase.regionserver.TestM >> e >> > >> mStore,org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyC >> o >> > ntrol,org.apache.hadoop.hbase.regionserver.TestSeekOptimizations,org.a >> > >> pache.hadoop.hbase.regionserver.TestSplitTransaction,org.apache.hadoop >> > .hbase.regionserver.TestStore,org.apache.hadoop.hbase.replication.Test >> > ReplicationSource >> > >> >> >> >> -- >> Todd Lipcon >> Software Engineer, Cloudera >
