This test that hangs looks like HBASE-3343. I can only make
TestMetaReaderEditor fail when running it on my laptop if I'm using
the "yes > /dev/null" trick. In all three logs I see this stack:
2010-12-20 16:41:53,328 WARN
[PostOpenDeployTasks:3d9e86152e5e75258c19d64d4ddb1a4a]
handler.OpenRegionHandler$PostOpenDeployTasksThread(195): Exception
running postOpenDeployTasks; region=3d9e86152e5e75258c19d64d4ddb1a4a
java.nio.channels.ClosedByInterruptException
at
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:184)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:518)
at
org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:192)
at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:311)
at
org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:865)
at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:732)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
at $Proxy12.get(Unknown Source)
at
org.apache.hadoop.hbase.catalog.MetaReader.readLocation(MetaReader.java:286)
at
org.apache.hadoop.hbase.catalog.MetaReader.readMetaLocation(MetaReader.java:262)
at
org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnection(CatalogTracker.java:279)
J-D
On Mon, Dec 20, 2010 at 2:58 PM, Apache Hudson Server
<[email protected]> wrote:
> See <https://hudson.apache.org/hudson/job/hbase-0.90/37/changes>
>
> Changes:
>
> [apurtell] HBASE-3377 Upgrade Jetty to 6.1.26
>
> [stack] HBASE-3323 OOME in master splitting logs
>
> ------------------------------------------
> [...truncated 2723 lines...]
> Running org.apache.hadoop.hbase.util.TestDefaultEnvironmentEdge
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
> Running org.apache.hadoop.hbase.rest.TestTransform
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.699 sec
> Running org.apache.hadoop.hbase.util.TestFSUtils
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.66 sec
> Running org.apache.hadoop.hbase.security.TestUser
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec
> Running org.apache.hadoop.hbase.master.TestMasterTransitions
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 89.378 sec
> Running org.apache.hadoop.hbase.regionserver.TestScanWildcardColumnTracker
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.121 sec
> Running org.apache.hadoop.hbase.regionserver.TestKeyValueSkipListSet
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 sec
> Running org.apache.hadoop.hbase.filter.TestPageFilter
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
> Running org.apache.hadoop.hbase.io.hfile.TestSeekTo
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.204 sec
> Running org.apache.hadoop.hbase.filter.TestColumnPaginationFilter
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.015 sec
> Running org.apache.hadoop.hbase.rest.TestStatusResource
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.058 sec
> Running org.apache.hadoop.hbase.executor.TestExecutorService
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.049 sec
> Running org.apache.hadoop.hbase.client.TestFromClientSide
> Tests run: 41, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 337.081 sec
> Running org.apache.hadoop.hbase.replication.TestReplication
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 211.499 sec
> Running org.apache.hadoop.hbase.regionserver.TestCompaction
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 130.281 sec
> Running org.apache.hadoop.hbase.filter.TestSingleColumnValueFilter
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.035 sec
> Running org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.986 sec
> Running org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServerArg
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec
> Running org.apache.hadoop.hbase.mapreduce.TestSimpleTotalOrderPartitioner
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.122 sec
> Running org.apache.hadoop.hbase.master.TestClockSkewDetection
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.041 sec
> Running org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 172.764 sec
> Running org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.075 sec
> Running org.apache.hadoop.hbase.client.replication.TestReplicationAdmin
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.44 sec
> Running org.apache.hadoop.hbase.regionserver.TestScanDeleteTracker
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.179 sec
> Running org.apache.hadoop.hbase.client.TestMetaScanner
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 50.668 sec
> Running org.apache.hadoop.hbase.metrics.TestMetricsMBeanBase
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 sec
> Running org.apache.hadoop.hbase.TestRegionRebalancing
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 77.188 sec
> Running org.apache.hadoop.hbase.filter.TestSingleColumnValueExcludeFilter
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec
> Running org.apache.hadoop.hbase.filter.TestInclusiveStopFilter
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
> Running org.apache.hadoop.hbase.rest.TestTableResource
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.9 sec
> Running org.apache.hadoop.hbase.TestHBaseTestingUtility
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.436 sec
> Running org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 66.219 sec
> Running org.apache.hadoop.hbase.rest.TestSchemaResource
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 56.111 sec
> Running org.apache.hadoop.hbase.regionserver.wal.TestWALReplay
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.935 sec
> Running org.apache.hadoop.hbase.regionserver.TestKeyValueScanFixture
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
> Running org.apache.hadoop.hbase.rest.model.TestTableListModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.139 sec
> Running org.apache.hadoop.hbase.rest.model.TestColumnSchemaModel
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.031 sec
> Running org.apache.hadoop.hbase.regionserver.TestStoreFile
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 34.68 sec
> Running org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.418 sec
> Running org.apache.hadoop.hbase.rest.model.TestScannerModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.141 sec
> Running org.apache.hadoop.hbase.regionserver.wal.TestHLogMethods
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.436 sec
> Running org.apache.hadoop.hbase.rest.TestVersionResource
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 44.831 sec
> Running org.apache.hadoop.hbase.filter.TestFilter
> Tests run: 18, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.946 sec
> Running org.apache.hadoop.hbase.regionserver.TestWideScanner
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.388 sec
> Running org.apache.hadoop.hbase.regionserver.TestSplitTransaction
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.986 sec
> Running org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.58 sec
> Running org.apache.hadoop.hbase.regionserver.TestGetClosestAtOrBefore
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.116 sec
> Running org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyControl
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.02 sec
> Running org.apache.hadoop.hbase.util.TestMergeTable
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.277 sec
> Running org.apache.hadoop.hbase.rest.model.TestVersionModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.145 sec
> Running org.apache.hadoop.hbase.rest.model.TestCellSetModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.15 sec
> Running org.apache.hadoop.hbase.rest.client.TestRemoteTable
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.566 sec
> Running org.apache.hadoop.hbase.rest.model.TestStorageClusterStatusModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.148 sec
> Running org.apache.hadoop.hbase.avro.TestAvroServer
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 84.54 sec
> Running org.apache.hadoop.hbase.TestFullLogReconstruction
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 75.509 sec
> Running org.apache.hadoop.hbase.TestSerialization
> Tests run: 19, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.075 sec
> Running org.apache.hadoop.hbase.regionserver.TestKeyValueHeap
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.13 sec
> Running org.apache.hadoop.hbase.regionserver.TestStoreScanner
> Tests run: 14, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 sec
> Running org.apache.hadoop.hbase.TestHMsg
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 sec
> Running org.apache.hadoop.hbase.io.hfile.TestReseekTo
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.236 sec
> Running org.apache.hadoop.hbase.regionserver.TestHRegionInfo
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
> Running org.apache.hadoop.hbase.util.TestIncrementingEnvironmentEdge
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 sec
> Running org.apache.hadoop.hbase.util.TestKeying
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec
> Running org.apache.hadoop.hbase.regionserver.TestScanner
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 58.677 sec
> Running org.apache.hadoop.hbase.util.TestHBaseFsck
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 38.289 sec
> Running org.apache.hadoop.hbase.master.TestRestartCluster
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 114.792 sec
> Running org.apache.hadoop.hbase.client.TestShell
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 74.326 sec
> Running org.apache.hadoop.hbase.rest.model.TestTableInfoModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.144 sec
> Running org.apache.hadoop.hbase.TestInfoServers
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.717 sec
> Running org.apache.hadoop.hbase.master.TestLogsCleaner
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.179 sec
> Running org.apache.hadoop.hbase.regionserver.TestMasterAddressManager
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.305 sec
> Running org.apache.hadoop.hbase.regionserver.TestQueryMatcher
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.157 sec
> Running org.apache.hadoop.hbase.zookeeper.TestZKTable
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.384 sec
> Running org.apache.hadoop.hbase.client.TestAdmin
> Tests run: 16, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 521.547 sec
> Running org.apache.hadoop.hbase.regionserver.TestPriorityCompactionQueue
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.074 sec
> Running org.apache.hadoop.hbase.util.TestMergeTool
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 42.37 sec
> Running org.apache.hadoop.hbase.TestAcidGuarantees
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 0.016 sec
> Running org.apache.hadoop.hbase.util.TestBase64
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.017 sec
> Running org.apache.hadoop.hbase.regionserver.TestMemStore
> Tests run: 20, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.418 sec
> Running org.apache.hadoop.hbase.rest.TestScannerResource
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 71.894 sec
> Running org.apache.hadoop.hbase.io.hfile.TestLruBlockCache
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.06 sec
> Running org.apache.hadoop.hbase.master.TestDeadServer
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.019 sec
> Running org.apache.hadoop.hbase.rest.TestGzipFilter
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.02 sec
> Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 158.222 sec
> Running org.apache.hadoop.hbase.TestKeyValue
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.025 sec
> Running org.apache.hadoop.hbase.client.TestMultiParallel
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 70.515 sec
> Running org.apache.hadoop.hbase.master.TestMaster
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 110.784 sec
> Running org.apache.hadoop.hbase.master.TestRollingRestart
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 79.547 sec
> Running org.apache.hadoop.hbase.client.TestResult
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
> Running org.apache.hadoop.hbase.client.TestHTablePool
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.772 sec
> Running org.apache.hadoop.hbase.rest.model.TestTableSchemaModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.157 sec
> Running org.apache.hadoop.hbase.util.TestByteBloomFilter
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.581 sec
> Running org.apache.hadoop.hbase.filter.TestFilterList
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.042 sec
> Running org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan
> Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 334.24 sec
> Running org.apache.hadoop.hbase.regionserver.TestHRegion
> Tests run: 52, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 35.052 sec
> Running org.apache.hadoop.hbase.master.TestMasterFailover
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 93.418 sec
> Running org.apache.hadoop.hbase.rest.model.TestTableRegionModel
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
> Running org.apache.hadoop.hbase.client.TestTimestampsFilter
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.597 sec
> Running org.apache.hadoop.hbase.master.TestLoadBalancer
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.195 sec
> Running org.apache.hadoop.hbase.util.TestRootPath
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec
> Running org.apache.hadoop.hbase.client.TestScannerTimeout
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 69.852 sec
> Running org.apache.hadoop.hbase.mapred.TestTableMapReduce
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 51.674 sec
> Running org.apache.hadoop.hbase.regionserver.wal.TestWALObserver
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.424 sec
> Running org.apache.hadoop.hbase.master.TestActiveMasterManager
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.519 sec
> Running
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.957 sec
> Running org.apache.hadoop.hbase.util.TestBytes
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.023 sec
> Running org.apache.hadoop.hbase.util.TestCompressionTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.034 sec
> Running org.apache.hadoop.hbase.client.TestGetRowVersions
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 55.827 sec
> Running org.apache.hadoop.hbase.filter.TestDependentColumnFilter
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.387 sec
> Running org.apache.hadoop.hbase.rest.model.TestCellModel
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.141 sec
> Running org.apache.hadoop.hbase.master.TestCatalogJanitor
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.506 sec
> Running org.apache.hadoop.hbase.catalog.TestCatalogTracker
> Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.557 sec
> Running org.apache.hadoop.hbase.replication.TestReplicationSource
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.899 sec
> Running org.apache.hadoop.hbase.rest.TestRowResource
> Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 85.001 sec
> Running org.apache.hadoop.hbase.io.hfile.TestHFile
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.372 sec
> Running org.apache.hadoop.hbase.catalog.TestMetaReaderEditor
> killed.
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] BUILD ERROR
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Error while executing forked tests.; nested exception is
> org.apache.maven.surefire.booter.shade.org.codehaus.plexus.util.cli.CommandLineException:
> Error while executing external command, process killed.
>
> Process timeout out after 900 seconds
> [INFO]
> ------------------------------------------------------------------------
> [INFO] For more information, run Maven with the -e switch
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 101 minutes 48 seconds
> [INFO] Finished at: Mon Dec 20 22:58:07 UTC 2010
> [INFO] Final Memory: 96M/1133M
> [INFO]
> ------------------------------------------------------------------------
> [locks-and-latches] Releasing all the locks
> [locks-and-latches] All the locks released
> Archiving artifacts
> Recording test results
>
>
