[
https://issues.apache.org/jira/browse/HBASE-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
nkeywal updated HBASE-4737:
---------------------------
Attachment: hbasetests.sh
The script launch first the small tests (list hardcoded until the categories
are set in the source code) and run the other tests with two parallel maven
instance.
Can replay the failed tests.
Still beta, but improve the build time by around 40% => less than 1 hour for
the whole suite test.
> Split the tests in small/medium/large; allow small tests to be ran in
> parallel within a single JVM
> --------------------------------------------------------------------------------------------------
>
> Key: HBASE-4737
> URL: https://issues.apache.org/jira/browse/HBASE-4737
> Project: HBase
> Issue Type: Improvement
> Components: test
> Affects Versions: 0.94.0
> Reporter: nkeywal
> Assignee: nkeywal
> Priority: Minor
> Attachments: 20111103_4737_pom.dummy.patch, 20111103_4737_pom.patch,
> 20111103_4737_pom.patch, 20111103_4737_pom.patch, 20111103_4737_pom.patch,
> 20111103_4737_pom.v2.patch, 20111103_4737_pom.v2.patch, hbasetests.sh
>
>
> 1) Split the tests in 3 categories
> - small: no cluster, less than 15s, can be run in parallel with other tests
> in a JVM
> - medium: <45s, no flaky, useful to detect bugs immediatly
> - large: remaining
>
> 2) Allow to run a subset: developpers should need to run only small and
> medium before submitting a patch
> - will need a surefire patch, see
> http://jira.codehaus.org/browse/SUREFIRE-329
> Small is the default. All other tests will have to be marked Medium or Large
> with a JUnit category.
> Proposed split:
> Small:122 classes, 479 methods, ~3 minutes when no //)
> Medium: 78 classes, 373 methods, ~23 minutes
> Large: 34 classes, 221 methods, ~60 minutes
> I will have to extract the methods that are today in large or medium but
> could be in small (typically io.hfile.TestHFileBlock#testBlockHeapSize), it
> will be done in a second step (and another JIRA).
> MEDIUM LIST (name; number of methods, time)
> org.apache.hadoop.hbase.avro.TestAvroServer 3 31.468
> org.apache.hadoop.hbase.catalog.TestCatalogTracker 8 4.174
> org.apache.hadoop.hbase.catalog.TestMetaReaderEditorNoCluster 1 3.888
> org.apache.hadoop.hbase.catalog.TestMetaReaderEditor 5 30.157
> org.apache.hadoop.hbase.client.replication.TestReplicationAdmin 1
> 0.762
> org.apache.hadoop.hbase.client.TestHCM 3 21.961
> org.apache.hadoop.hbase.client.TestHTablePool 18 26.274
> org.apache.hadoop.hbase.client.TestHTableUtil 2 16.997
> org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD 3 24.629
> org.apache.hadoop.hbase.client.TestMetaScanner 1 16.365
> org.apache.hadoop.hbase.client.TestMultiParallel 10 34.077
> org.apache.hadoop.hbase.client.TestTimestampsFilter 3 27.547
> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol 44 16.834
> org.apache.hadoop.hbase.coprocessor.TestClassLoading 5 31.346
> org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint 2 32.736
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort
> 1 13.874
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove
> 1 16.923
> org.apache.hadoop.hbase.coprocessor.TestMasterObserver 3 29.97
> org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass 2 14.976
> org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface 5
> 33.353
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort
> 1 16.596
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove
> 1 18.183
> org.apache.hadoop.hbase.coprocessor.TestWALObserver 3 19.373
> org.apache.hadoop.hbase.filter.TestColumnRangeFilter 1 19.045
> org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache 5 24.294
> org.apache.hadoop.hbase.io.hfile.slab.TestSlabCache 7 19.818
> org.apache.hadoop.hbase.io.hfile.TestHFileBlock 7 25.226
> org.apache.hadoop.hbase.io.hfile.TestLruBlockCache 7 0.343
> org.apache.hadoop.hbase.mapreduce.TestImportTsv 8 40.391
> org.apache.hadoop.hbase.master.TestActiveMasterManager 2 0.724
> org.apache.hadoop.hbase.master.TestHMasterRPCException 1 1.17
> org.apache.hadoop.hbase.master.TestLogsCleaner 1 2.953
> org.apache.hadoop.hbase.master.TestMaster 1 18.918
> org.apache.hadoop.hbase.master.TestOpenedRegionHandler 2 20.57
> org.apache.hadoop.hbase.master.TestSplitLogManager 10 13.979
> org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion 3 21.675
> org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler 3
> 0.887
> org.apache.hadoop.hbase.regionserver.TestBlocksRead 4 1.42
> org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter 3 22.694
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed 3 29.764
> org.apache.hadoop.hbase.regionserver.TestHRegion 57 28.552
> org.apache.hadoop.hbase.regionserver.TestMasterAddressManager 1 0.525
> org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner 6 19.568
> org.apache.hadoop.hbase.regionserver.TestRpcMetrics 1 2.028
> org.apache.hadoop.hbase.regionserver.TestSeekOptimizations 6 3.031
> org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol 6 20.087
> org.apache.hadoop.hbase.regionserver.TestSplitLogWorker 5 2.062
> org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary 1
> 16.88
> org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort 1 22.545
> org.apache.hadoop.hbase.regionserver.wal.TestWALReplay 5 29.678
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink 5
> 23.19
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager
> 1 0.918
> org.apache.hadoop.hbase.replication.TestReplicationPeer 1 25.518
> org.apache.hadoop.hbase.rest.client.TestRemoteAdmin 1 16.141
> org.apache.hadoop.hbase.rest.client.TestRemoteTable 3 17.827
> org.apache.hadoop.hbase.rest.TestGzipFilter 3 17.404
> org.apache.hadoop.hbase.rest.TestMultiRowResource 3 17.314
> org.apache.hadoop.hbase.rest.TestRowResource 11 17.73
> org.apache.hadoop.hbase.rest.TestScannerResource 5 26.868
> org.apache.hadoop.hbase.rest.TestScannersWithFilters 10 20.774
> org.apache.hadoop.hbase.rest.TestSchemaResource 2 22.441
> org.apache.hadoop.hbase.rest.TestStatusResource 2 14.884
> org.apache.hadoop.hbase.rest.TestTableResource 8 25.658
> org.apache.hadoop.hbase.rest.TestTransform 1 15.966
> org.apache.hadoop.hbase.rest.TestVersionResource 7 19.124
> org.apache.hadoop.hbase.TestGlobalMemStoreSize 1 18.154
> org.apache.hadoop.hbase.TestInfoServers 2 14.9
> org.apache.hadoop.hbase.TestMultiVersions 3 27.945
> org.apache.hadoop.hbase.thrift.TestThriftServer 1 20.072
> org.apache.hadoop.hbase.util.TestDefaultEnvironmentEdge 1 0.055
> org.apache.hadoop.hbase.util.TestEnvironmentEdgeManager 2 0.215
> org.apache.hadoop.hbase.util.TestFSUtils 2 19.988
> org.apache.hadoop.hbase.util.TestHBaseFsck 6 41.133
> org.apache.hadoop.hbase.util.TestIncrementingEnvironmentEdge 1 0.054
> org.apache.hadoop.hbase.util.TestMergeTable 1 27.125
> org.apache.hadoop.hbase.util.TestRegionSplitter 4 28.845
> org.apache.hadoop.hbase.zookeeper.TestZKTable 1 0.529
> org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker 2 0.569
> LARGE
> org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster 1 84.463
> org.apache.hadoop.hbase.client.TestAdmin 33 369.833
> org.apache.hadoop.hbase.client.TestFromClientSide 49 154.41
> org.apache.hadoop.hbase.client.TestMultipleTimestamps 8 66.497
> org.apache.hadoop.hbase.client.TestScannerTimeout 4 47.309
> org.apache.hadoop.hbase.client.TestShell 1 69.65
> org.apache.hadoop.hbase.mapred.TestTableInputFormat 6 24.39
> org.apache.hadoop.hbase.mapred.TestTableMapReduce 1 63.846
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat 8 152.609
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery
> 3 74.628
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles 6 35.25
> org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan 11 516.619
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce 1 92.144
> org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed 1 42.761
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting 4 112.606
> org.apache.hadoop.hbase.master.TestMasterFailover 4 76.658
> org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable 1
> 20.86
> org.apache.hadoop.hbase.master.TestMasterTransitions 3 22.043
> org.apache.hadoop.hbase.master.TestRestartCluster 2 36.325
> org.apache.hadoop.hbase.master.TestRollingRestart 1 54.738
> org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction 1
> 40.935
> org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad 1
> 47.073
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster 4
> 43.359
> org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit 28 230.379
> org.apache.hadoop.hbase.regionserver.wal.TestHLog 9 59.889
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling 3 310.537
> org.apache.hadoop.hbase.replication.TestMasterReplication 2 75.346
> org.apache.hadoop.hbase.replication.TestMultiSlaveReplication 1 47.873
> org.apache.hadoop.hbase.replication.TestReplication 7 163.158
> org.apache.hadoop.hbase.TestFullLogReconstruction 1 57.742
> org.apache.hadoop.hbase.TestHBaseTestingUtility 7 73.654
> org.apache.hadoop.hbase.TestRegionRebalancing 1 58.25
> org.apache.hadoop.hbase.TestZooKeeper 7 86.267
> org.apache.hadoop.hbase.util.TestMergeTool 1 257.223
> SMALL
> Everyting else
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
