That sounds perfect :-). For JUnit, there is nothing specific I used the script provided with JUnit (build_tests.sh). For surefire, I had to use -DskipTests to get it installed, even before my changes. Except this (!) it's pure standard maven.
Then there are two use cases: - short term, we just need to change the versions and the repository list in the .pom to see how it behaves. - medium term (3/4 days) we should publish the category split, I created 4781 to put the .pom I am currently testing. Thanks for your help! On Mon, Nov 14, 2011 at 8:47 PM, Gary Helmling <ghelml...@gmail.com> wrote: > Hi N, > > I can push up the modified surefire and junit build with appropriate > versions. > > Is there anything special to either build that would complicate > building with the supplied patches? If not, then I can do the builds > and push up the artifacts. Then we coordinate the testing. How does > that sound? > > --gh > > > On Mon, Nov 14, 2011 at 5:51 AM, N Keywal <nkey...@gmail.com> wrote: > > Hi Gary, > > > > The hbase specific versions for surefire & junit are ready (see > HBASE-4763 > > for all the details). > > Would it be possible to use your repository? What would you need from me? > > I've never deployed anything in a maven repo. Surefire works with maven, > so > > it should be easy, JUnit does not, so there could be more work to do. > > > > I copy the dev list for information. > > > > Regards, > > > > N. > > > > > > ---------- Forwarded message ---------- > > From: Gary Helmling <ghelml...@gmail.com> > > Date: Thu, Oct 20, 2011 at 11:50 PM > > Subject: Re: [unit tests] small tests vs. others > > To: dev@hbase.apache.org > > > > > > +1 on small/medium/large > > > > I have a test mvn repo up on people.apache.org that we can use for a > > patched surefire build, if that helps. > > > > > > On Thu, Oct 20, 2011 at 2:33 PM, Jonathan Gray <jg...@fb.com> wrote: > >> +1 on small/medium/large. > >> > >>> -----Original Message----- > >>> From: Todd Lipcon [mailto:t...@cloudera.com] > >>> Sent: Thursday, October 20, 2011 1:49 PM > >>> To: dev@hbase.apache.org > >>> Subject: Re: [unit tests] small tests vs. others > >>> > >>> +1 on separating into small/medium/large. One of the requirements of > >>> "small" should be that it can run parallel with other tests and in > nofork > >>> mode. > >>> > >>> See SUREFIRE-329 for the necessary surefire patch. Until that's > >>> integrated > >>> upstream we'd have to have our own patched surefire :( > >>> > >>> -Todd > >>> > >>> On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <nkey...@gmail.com> wrote: > >>> > Hi, > >>> > > >>> > I've done a stat on the tests we have. Many of them are actually > quite > >>> > simple: There are 226 test classes in HBase. 100 of them does not > >>> > use a cluster, and takes less than 2 seconds to execute. 24 do not > use > >>> > a cluster but need between 2 and 40 seconds. > >>> > > >>> > I tried to execute these with different maven settings. > >>> > fork: once or always, i.e. all the tests are executed in the same JVM > >>> > or there is a new JVM created for every test class parallel; > >>> > deactivated or activated, with 3 threads and a per class > >>> > parallelisation. > >>> > > >>> > The 100 tests executes for the whole maven stuff, including the > >>> > initial dependency & task analysis fork always: 1m50s fork once: 30 > >>> > seconds > >>> > parallel: 30s (=> no difference). > >>> > > >>> > It works quite well, except with TestLruBlockCache that I had to > >>> > remove. > >>> > > >>> > I tried as well with the 24 'bigger' ones but no cluster, for the > >>> > following > >>> > results: > >>> > fork always: 7m36s > >>> > parallel: 2m32s: 3 times faster. > >>> > > >>> > However, I had to remove many of them, there are some conflicts that > I > >>> > have not solved (nor tried to). > >>> > > >>> > The fact that nearly 50% of the tests can be executed in 1 minutes is > >>> > interesting imho. > >>> > > >>> > That makes me wonder if we should try to get a separation of the > tests > >>> > in 3 parts, small/medium/large as of > >>> > > http://googletesting.blogspot.com/2009/07/software-testing-categorizat > >>> > ion.html > >>> > > >>> > small would be: > >>> > - less than two seconds, can be executed in the same jvm > >>> > - all of them: less than 2 minute > >>> > - no cluster, nothing that can break the environment > >>> > - run by the developers as a part of the build > >>> > > >>> > medium: > >>> > - less than 40 seconds > >>> > - all of them: would last less than 30 minutes, ideally 15 minutes > >>> > - run in an isolated jvm > >>> > - run explicitly by the developers before submitting a fix > >>> > > >>> > large: > >>> > - run in an isolated jvm > >>> > - Should last less than 45minutes > >>> > - run on the central built or explicitly by the developers > >>> > - break the built when it fails. > >>> > > >>> > Only one issue: it's not directly possible with maven, but there are > >>> > known workarounds :-) > >>> > > >>> > > >>> > What do you think? > >>> > > >>> > > >>> > Below the tests set for the 120 tests that seems to work altogether. > >>> > > >>> > > >>> > mvn test > >>> > - > >>> Dtest=org.apache.hadoop.hbase.avro.TestAvroUtil,org.apache.hadoop.hba > >>> > > se.client.TestAttributes,org.apache.hadoop.hbase.client.TestGet,org.ap > >>> > > ache.hadoop.hbase.client.TestOperation,org.apache.hadoop.hbase.client. > >>> > > TestResult,org.apache.hadoop.hbase.client.TestScan,org.apache.hadoop.h > >>> > > >>> base.coprocessor.TestRegionObserverStacking,org.apache.hadoop.hbase.fi > >>> > > lter.TestBitComparator,org.apache.hadoop.hbase.filter.TestColumnPagina > >>> > > tionFilter,org.apache.hadoop.hbase.filter.TestDependentColumnFilter,or > >>> > g.apache.hadoop.hbase.filter.TestFilterList, > org.apache.hadoop.hbase.fi > >>> > > lter.TestInclusiveStopFilter,org.apache.hadoop.hbase.filter.TestPageFi > >>> > > lter,org.apache.hadoop.hbase.filter.TestParseFilter,org.apache.hadoop. > >>> > > hbase.filter.TestPrefixFilter,org.apache.hadoop.hbase.filter.TestRando > >>> > > >>> mRowFilter,org.apache.hadoop.hbase.filter.TestSingleColumnValueExclude > >>> > > Filter,org.apache.hadoop.hbase.filter.TestSingleColumnValueFilter,org. > >>> > > apache.hadoop.hbase.io.hfile.slab.TestSlabCache,org.apache.hadoop.hbas > >>> > > e.io.hfile.slab.TestSlab,org.apache.hadoop.hbase.io.hfile.TestBlockCac > >>> > > >>> heColumnFamilySummary,org.apache.hadoop.hbase.io.hfile.TestCachedBloc > >>> k > >>> > > Queue,org.apache.hadoop.hbase.io.hfile.TestFixedFileTrailer,org.apache > >>> > > .hadoop.hbase.io.hfile.TestHFileReaderV1,org.apache.hadoop.hbase.io.hf > >>> > > ile.TestHFile,org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2,org.a > >>> > > >>> pache.hadoop.hbase.io.hfile.TestReseekTo,org.apache.hadoop.hbase.io.hf > >>> > > ile.TestSeekTo,org.apache.hadoop.hbase.io.TestHalfStoreFileReader,org. > >>> > > >>> apache.hadoop.hbase.io.TestHbaseObjectWritable,org.apache.hadoop.hbas > >>> e > >>> > > >>> .io.TestHeapSize,org.apache.hadoop.hbase.io.TestImmutableBytesWritable > >>> > > >>> ,org.apache.hadoop.hbase.mapreduce.TestSimpleTotalOrderPartitioner,org > >>> > > >>> .apache.hadoop.hbase.mapreduce.TestTableSplit,org.apache.hadoop.hbase > >>> . > >>> > > >>> master.TestClockSkewDetection,org.apache.hadoop.hbase.master.TestDea > >>> dS > >>> > > >>> erver,org.apache.hadoop.hbase.master.TestDefaultLoadBalancer,org.apach > >>> > > >>> e.hadoop.hbase.master.TestMasterStatusServlet,org.apache.hadoop.hbase. > >>> > > >>> metrics.TestMetricsMBeanBase,org.apache.hadoop.hbase.monitoring.Test > >>> Me > >>> > > >>> moryBoundedLogMessageBuffer,org.apache.hadoop.hbase.monitoring.Test > >>> Tas > >>> > > >>> kMonitor,org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionH > >>> > > >>> andler,org.apache.hadoop.hbase.regionserver.TestCompactSelection,org.a > >>> > > pache.hadoop.hbase.regionserver.TestExplicitColumnTracker,org.apache.h > >>> > > >>> adoop.hbase.regionserver.TestHRegionInfo,org.apache.hadoop.hbase.regio > >>> > > >>> nserver.TestKeyValueHeap,org.apache.hadoop.hbase.regionserver.TestKey > >>> V > >>> > > >>> alueScanFixture,org.apache.hadoop.hbase.regionserver.TestKeyValueSkipL > >>> > > >>> istSet,org.apache.hadoop.hbase.regionserver.TestMemStoreLAB,org.apach > >>> e > >>> > > >>> .hadoop.hbase.regionserver.TestMinVersions,org.apache.hadoop.hbase.reg > >>> > > >>> ionserver.TestQueryMatcher,org.apache.hadoop.hbase.regionserver.TestRe > >>> > > gionSplitPolicy,org.apache.hadoop.hbase.regionserver.TestResettingCoun > >>> > > ters,org.apache.hadoop.hbase.regionserver.TestRSStatusServlet,org.apac > >>> > > >>> he.hadoop.hbase.regionserver.TestScanDeleteTracker,org.apache.hadoop.h > >>> > > >>> base.regionserver.TestScanWildcardColumnTracker,org.apache.hadoop.hbas > >>> > > >>> e.regionserver.TestScanWithBloomError,org.apache.hadoop.hbase.regionse > >>> > > >>> rver.TestStoreScanner,org.apache.hadoop.hbase.regionserver.wal.TestHLo > >>> > > >>> gBench,org.apache.hadoop.hbase.regionserver.wal.TestHLogMethods,org.a > >>> p > >>> > > >>> ache.hadoop.hbase.regionserver.wal.TestWALActionsListener,org.apache.h > >>> > > >>> adoop.hbase.rest.model.TestCellModel,org.apache.hadoop.hbase.rest.mod > >>> e > >>> > > >>> l.TestCellSetModel,org.apache.hadoop.hbase.rest.model.TestColumnSchem > >>> a > >>> > > >>> Model,org.apache.hadoop.hbase.rest.model.TestRowModelo,rg.apache.ha > >>> doo > >>> > > >>> p.hbase.rest.model.TestScannerModel,org.apache.hadoop.hbase.rest.mod > >>> el > >>> > > >>> .TestStorageClusterStatusModel,org.apache.hadoop.hbase.rest.model.Test > >>> > > >>> StorageClusterVersionModel,org.apache.hadoop.hbase.rest.model.TestTabl > >>> > > >>> eInfoModel,org.apache.hadoop.hbase.rest.model.TestTableListModel,org.a > >>> > > >>> pache.hadoop.hbase.rest.model.TestTableRegionModel,org.apache.hadoop > >>> .h > >>> > > >>> base.rest.model.TestTableSchemaModel,org.apache.hadoop.hbase.rest.mo > >>> de > >>> > > l.TestVersionModel,org.apache.hadoop.hbase.security.TestUser,org.apach > >>> > > >>> e.hadoop.hbase.TestAcidGuarantees,org.apache.hadoop.hbase.TestCompar > >>> e, > >>> > > >>> org.apache.hadoop.hbase.TestHRegionLocation,org.apache.hadoop.hbase.T > >>> e > >>> > > >>> stHServerAddress,org.apache.hadoop.hbase.TestHServerInfo,org.apache.ha > >>> > > doop.hbase.TestKeyValue,org.apache.hadoop.hbase.TestSerialization,org. > >>> > > >>> apache.hadoop.hbase.TestServerName,org.apache.hadoop.hbase.util.TestB > >>> a > >>> > > se64,org.apache.hadoop.hbase.util.TestByteBloomFilter,org.apache.hadoo > >>> > > p.hbase.util.TestBytes,org.apache.hadoop.hbase.util.TestCompressionTes > >>> > > t,org.apache.hadoop.hbase.util.TestFSTableDescriptors,org.apache.hadoo > >>> > > p.hbase.util.TestHBaseFsckComparator,org.apache.hadoop.hbase.util.Test > >>> > > >>> IncrementingEnvironmentEdge,org.apache.hadoop.hbase.util.TestKeying,or > >>> > > >>> g.apache.hadoop.hbase.util.TestPoolMap,org.apache.hadoop.hbase.util.Te > >>> > > >>> stRegionSplitCalculator,org.apache.hadoop.hbase.zookeeper.TestHQuorum > >>> P > >>> > > >>> eer,org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServerArg,org > >>> .a > >>> > > >>> pache.hadoop.hbase.coprocessor.TestCoprocessorInterface,org.apache.had > >>> > > oop.hbase.executor.TestExecutorService,org.apache.hadoop.hbase.filter. > >>> > > TestColumnPrefixFilter,org.apache.hadoop.hbase.filter.TestFilter,org.a > >>> > > pache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter,org.apache.ha > >>> > > doop.hbase.io.hfile.slab.TestSingleSizeCache,org.apache.hadoop.hbase.i > >>> > > o.hfile.TestCacheOnWrite,org.apache.hadoop.hbase.io.hfile.TestHFileBlo > >>> > > ckIndex,org.apache.hadoop.hbase.io.hfile.TestHFileBlock,org.apache.had > >>> > > oop.hbase.io.hfile.TestHFilePerformance,org.apache.hadoop.hbase.io.hfi > >>> > > >>> le.TestHFileSeek,org.apache.hadoop.hbase.ipc.TestDelayedRpc,org.apache > >>> > > >>> .hadoop.hbase.master.TestCatalogJanitor,org.apache.hadoop.hbase.region > >>> > > >>> server.TestAtomicOperation,org.apache.hadoop.hbase.regionserver.TestM > >>> e > >>> > > >>> mStore,org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyC > >>> o > >>> > > ntrol,org.apache.hadoop.hbase.regionserver.TestSeekOptimizations,org.a > >>> > > >>> pache.hadoop.hbase.regionserver.TestSplitTransaction,org.apache.hadoop > >>> > > .hbase.regionserver.TestStore,org.apache.hadoop.hbase.replication.Test > >>> > ReplicationSource > >>> > > >>> > >>> > >>> > >>> -- > >>> Todd Lipcon > >>> Software Engineer, Cloudera > >> > > > > >
