Sounds good Nicolas. St.Ack
On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <nkey...@gmail.com> wrote: > Hi, > > I've done a stat on the tests we have. Many of them are actually quite > simple: There are 226 test classes in HBase. 100 of them does not use a > cluster, and takes less than 2 seconds to execute. 24 do not use a cluster > but need between 2 and 40 seconds. > > I tried to execute these with different maven settings. > fork: once or always, i.e. all the tests are executed in the same JVM or > there is a new JVM created for every test class > parallel; deactivated or activated, with 3 threads and a per class > parallelisation. > > The 100 tests executes for the whole maven stuff, including the initial > dependency & task analysis > fork always: 1m50s > fork once: 30 seconds > parallel: 30s (=> no difference). > > It works quite well, except with TestLruBlockCache that I had to remove. > > I tried as well with the 24 'bigger' ones but no cluster, for the following > results: > fork always: 7m36s > parallel: 2m32s: 3 times faster. > > However, I had to remove many of them, there are some conflicts that I have > not solved (nor tried to). > > The fact that nearly 50% of the tests can be executed in 1 minutes is > interesting imho. > > That makes me wonder if we should try to get a separation of the tests in 3 > parts, small/medium/large as of > http://googletesting.blogspot.com/2009/07/software-testing-categorization.html > > small would be: > - less than two seconds, can be executed in the same jvm > - all of them: less than 2 minute > - no cluster, nothing that can break the environment > - run by the developers as a part of the build > > medium: > - less than 40 seconds > - all of them: would last less than 30 minutes, ideally 15 minutes > - run in an isolated jvm > - run explicitly by the developers before submitting a fix > > large: > - run in an isolated jvm > - Should last less than 45minutes > - run on the central built or explicitly by the developers > - break the built when it fails. > > Only one issue: it's not directly possible with maven, but there are known > workarounds :-) > > > What do you think? > > > Below the tests set for the 120 tests that seems to work altogether. > > > mvn test > -Dtest=org.apache.hadoop.hbase.avro.TestAvroUtil,org.apache.hadoop.hbase.client.TestAttributes,org.apache.hadoop.hbase.client.TestGet,org.apache.hadoop.hbase.client.TestOperation,org.apache.hadoop.hbase.client.TestResult,org.apache.hadoop.hbase.client.TestScan,org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking,org.apache.hadoop.hbase.filter.TestBitComparator,org.apache.hadoop.hbase.filter.TestColumnPaginationFilter,org.apache.hadoop.hbase.filter.TestDependentColumnFilter,org.apache.hadoop.hbase.filter.TestFilterList,org.apache.hadoop.hbase.filter.TestInclusiveStopFilter,org.apache.hadoop.hbase.filter.TestPageFilter,org.apache.hadoop.hbase.filter.TestParseFilter,org.apache.hadoop.hbase.filter.TestPrefixFilter,org.apache.hadoop.hbase.filter.TestRandomRowFilter,org.apache.hadoop.hbase.filter.TestSingleColumnValueExcludeFilter,org.apache.hadoop.hbase.filter.TestSingleColumnValueFilter,org.apache.hadoop.hbase.io.hfile.slab.TestSlabCache,org.apache.hadoop.hbase.io.hfile.slab.TestSlab,org.apache.hadoop.hbase.io.hfile.TestBlockCacheColumnFamilySummary,org.apache.hadoop.hbase.io.hfile.TestCachedBlockQueue,org.apache.hadoop.hbase.io.hfile.TestFixedFileTrailer,org.apache.hadoop.hbase.io.hfile.TestHFileReaderV1,org.apache.hadoop.hbase.io.hfile.TestHFile,org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2,org.apache.hadoop.hbase.io.hfile.TestReseekTo,org.apache.hadoop.hbase.io.hfile.TestSeekTo,org.apache.hadoop.hbase.io.TestHalfStoreFileReader,org.apache.hadoop.hbase.io.TestHbaseObjectWritable,org.apache.hadoop.hbase.io.TestHeapSize,org.apache.hadoop.hbase.io.TestImmutableBytesWritable,org.apache.hadoop.hbase.mapreduce.TestSimpleTotalOrderPartitioner,org.apache.hadoop.hbase.mapreduce.TestTableSplit,org.apache.hadoop.hbase.master.TestClockSkewDetection,org.apache.hadoop.hbase.master.TestDeadServer,org.apache.hadoop.hbase.master.TestDefaultLoadBalancer,org.apache.hadoop.hbase.master.TestMasterStatusServlet,org.apache.hadoop.hbase.metrics.TestMetricsMBeanBase,org.apache.hadoop.hbase.monitoring.TestMemoryBoundedLogMessageBuffer,org.apache.hadoop.hbase.monitoring.TestTaskMonitor,org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionHandler,org.apache.hadoop.hbase.regionserver.TestCompactSelection,org.apache.hadoop.hbase.regionserver.TestExplicitColumnTracker,org.apache.hadoop.hbase.regionserver.TestHRegionInfo,org.apache.hadoop.hbase.regionserver.TestKeyValueHeap,org.apache.hadoop.hbase.regionserver.TestKeyValueScanFixture,org.apache.hadoop.hbase.regionserver.TestKeyValueSkipListSet,org.apache.hadoop.hbase.regionserver.TestMemStoreLAB,org.apache.hadoop.hbase.regionserver.TestMinVersions,org.apache.hadoop.hbase.regionserver.TestQueryMatcher,org.apache.hadoop.hbase.regionserver.TestRegionSplitPolicy,org.apache.hadoop.hbase.regionserver.TestResettingCounters,org.apache.hadoop.hbase.regionserver.TestRSStatusServlet,org.apache.hadoop.hbase.regionserver.TestScanDeleteTracker,org.apache.hadoop.hbase.regionserver.TestScanWildcardColumnTracker,org.apache.hadoop.hbase.regionserver.TestScanWithBloomError,org.apache.hadoop.hbase.regionserver.TestStoreScanner,org.apache.hadoop.hbase.regionserver.wal.TestHLogBench,org.apache.hadoop.hbase.regionserver.wal.TestHLogMethods,org.apache.hadoop.hbase.regionserver.wal.TestWALActionsListener,org.apache.hadoop.hbase.rest.model.TestCellModel,org.apache.hadoop.hbase.rest.model.TestCellSetModel,org.apache.hadoop.hbase.rest.model.TestColumnSchemaModel,org.apache.hadoop.hbase.rest.model.TestRowModelo,rg.apache.hadoop.hbase.rest.model.TestScannerModel,org.apache.hadoop.hbase.rest.model.TestStorageClusterStatusModel,org.apache.hadoop.hbase.rest.model.TestStorageClusterVersionModel,org.apache.hadoop.hbase.rest.model.TestTableInfoModel,org.apache.hadoop.hbase.rest.model.TestTableListModel,org.apache.hadoop.hbase.rest.model.TestTableRegionModel,org.apache.hadoop.hbase.rest.model.TestTableSchemaModel,org.apache.hadoop.hbase.rest.model.TestVersionModel,org.apache.hadoop.hbase.security.TestUser,org.apache.hadoop.hbase.TestAcidGuarantees,org.apache.hadoop.hbase.TestCompare,org.apache.hadoop.hbase.TestHRegionLocation,org.apache.hadoop.hbase.TestHServerAddress,org.apache.hadoop.hbase.TestHServerInfo,org.apache.hadoop.hbase.TestKeyValue,org.apache.hadoop.hbase.TestSerialization,org.apache.hadoop.hbase.TestServerName,org.apache.hadoop.hbase.util.TestBase64,org.apache.hadoop.hbase.util.TestByteBloomFilter,org.apache.hadoop.hbase.util.TestBytes,org.apache.hadoop.hbase.util.TestCompressionTest,org.apache.hadoop.hbase.util.TestFSTableDescriptors,org.apache.hadoop.hbase.util.TestHBaseFsckComparator,org.apache.hadoop.hbase.util.TestIncrementingEnvironmentEdge,org.apache.hadoop.hbase.util.TestKeying,org.apache.hadoop.hbase.util.TestPoolMap,org.apache.hadoop.hbase.util.TestRegionSplitCalculator,org.apache.hadoop.hbase.zookeeper.TestHQuorumPeer,org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServerArg,org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface,org.apache.hadoop.hbase.executor.TestExecutorService,org.apache.hadoop.hbase.filter.TestColumnPrefixFilter,org.apache.hadoop.hbase.filter.TestFilter,org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter,org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache,org.apache.hadoop.hbase.io.hfile.TestCacheOnWrite,org.apache.hadoop.hbase.io.hfile.TestHFileBlockIndex,org.apache.hadoop.hbase.io.hfile.TestHFileBlock,org.apache.hadoop.hbase.io.hfile.TestHFilePerformance,org.apache.hadoop.hbase.io.hfile.TestHFileSeek,org.apache.hadoop.hbase.ipc.TestDelayedRpc,org.apache.hadoop.hbase.master.TestCatalogJanitor,org.apache.hadoop.hbase.regionserver.TestAtomicOperation,org.apache.hadoop.hbase.regionserver.TestMemStore,org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl,org.apache.hadoop.hbase.regionserver.TestSeekOptimizations,org.apache.hadoop.hbase.regionserver.TestSplitTransaction,org.apache.hadoop.hbase.regionserver.TestStore,org.apache.hadoop.hbase.replication.TestReplicationSource >
