See <https://builds.apache.org/job/HBase-1.3-IT/it.test=IntegrationTestAcidGuarantees,jdk=latest1.8,label=Hadoop/183/changes>
Changes: [enis] HBASE-14474 DeadLock in RpcClientImpl.Connection.close() ------------------------------------------ [...truncated 1846 lines...] [INFO] Apache HBase - Shell .............................. SUCCESS [0.884s] [INFO] Apache HBase - Integration Tests .................. FAILURE [30:01.949s] [INFO] Apache HBase - Examples ........................... SKIPPED [INFO] Apache HBase - External Block Cache ............... SKIPPED [INFO] Apache HBase - Assembly ........................... SKIPPED [INFO] Apache HBase - Shaded ............................. SKIPPED [INFO] Apache HBase - Shaded - Client .................... SKIPPED [INFO] Apache HBase - Shaded - Server .................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 31:33.485s [INFO] Finished at: Fri Sep 25 22:00:28 UTC 2015 [INFO] Final Memory: 168M/2648M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-failsafe-plugin:2.18.1:verify (verify) on project hbase-it: There was a timeout or other error in the fork -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hbase-it Build step 'Invoke top-level Maven targets' marked build as failure Performing Post build task... Match found for :.* : True Logical operation result is TRUE Running script : ZOMBIE_TESTS_COUNT=`jps | grep surefirebooter | wc -l` if [[ $ZOMBIE_TESTS_COUNT != 0 ]] ; then #It seems sometimes the tests are not dying immediately. Let's give them 10s echo "Suspicious java process found - waiting 10s to see if there are just slow to stop" sleep 10 ZOMBIE_TESTS_COUNT=`jps | grep surefirebooter | wc -l` if [[ $ZOMBIE_TESTS_COUNT != 0 ]] ; then echo "There are $ZOMBIE_TESTS_COUNT zombie tests, they should have been killed by surefire but survived" echo "************ BEGIN zombies jstack extract" ZB_STACK=`jps | grep surefirebooter | cut -d ' ' -f 1 | xargs -n 1 jstack | grep ".test" | grep "\.java"` jps | grep surefirebooter | cut -d ' ' -f 1 | xargs -n 1 jstack echo "************ END zombies jstack extract" JIRA_COMMENT="$JIRA_COMMENT {color:red}-1 core zombie tests{color}. There are ${ZOMBIE_TESTS_COUNT} zombie test(s): ${ZB_STACK}" BAD=1 jps | grep surefirebooter | cut -d ' ' -f 1 | xargs kill -9 else echo "We're ok: there is no zombie test, but some tests took some time to stop" fi else echo "We're ok: there is no zombie test" fi [Hadoop] $ /bin/bash -xe /tmp/hudson5413880829488037253.sh ++ wc -l ++ grep surefirebooter ++ jps + ZOMBIE_TESTS_COUNT=2 + [[ 2 != 0 ]] + echo 'Suspicious java process found - waiting 10s to see if there are just slow to stop' Suspicious java process found - waiting 10s to see if there are just slow to stop + sleep 10 ++ grep surefirebooter ++ jps ++ wc -l + ZOMBIE_TESTS_COUNT=1 + [[ 1 != 0 ]] + echo 'There are 1 zombie tests, they should have been killed by surefire but survived' There are 1 zombie tests, they should have been killed by surefire but survived + echo '************ BEGIN zombies jstack extract' ************ BEGIN zombies jstack extract ++ grep '\.java' ++ grep .test ++ xargs -n 1 jstack ++ cut -d ' ' -f 1 ++ grep surefirebooter ++ jps + ZB_STACK=' at org.apache.hadoop.hdfs.web.TestWebHdfsTokens.testLazyTokenFetchForSWebhdfs(TestWebHdfsTokens.java:221)' + jps + grep surefirebooter + cut -d ' ' -f 1 + xargs -n 1 jstack 2015-09-25 22:00:41 Full thread dump Java HotSpot(TM) 64-Bit Server VM (24.55-b03 mixed mode): "Attach Listener" daemon prio=10 tid=0x00007fa1fc001000 nid=0x220e waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner" daemon prio=10 tid=0x0000000000da4800 nid=0x21a0 in Object.wait() [0x00007fa20011a000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000007b68109b0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:135) - locked <0x00000007b68109b0> (a java.lang.ref.ReferenceQueue$Lock) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at org.apache.hadoop.fs.FileSystem$Statistics$StatisticsDataReferenceCleaner.run(FileSystem.java:3116) at java.lang.Thread.run(Thread.java:745) "process reaper" daemon prio=10 tid=0x00007fa20c9e4800 nid=0x219c waiting on condition [0x00007fa200254000] java.lang.Thread.State: TIMED_WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x00000007b6810a68> (a java.util.concurrent.SynchronousQueue$TransferStack) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:226) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:359) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:942) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) "Service Thread" daemon prio=10 tid=0x00007fa20c291800 nid=0x2197 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread1" daemon prio=10 tid=0x00007fa20c28e800 nid=0x2196 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "C2 CompilerThread0" daemon prio=10 tid=0x00007fa20c28b800 nid=0x2195 waiting on condition [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Signal Dispatcher" daemon prio=10 tid=0x00007fa20c281800 nid=0x2194 runnable [0x0000000000000000] java.lang.Thread.State: RUNNABLE "Finalizer" daemon prio=10 tid=0x00007fa20c26b000 nid=0x2193 runnable [0x00007fa2013d2000] java.lang.Thread.State: RUNNABLE at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:133) at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:151) at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:189) "Reference Handler" daemon prio=10 tid=0x00007fa20c267000 nid=0x2192 in Object.wait() [0x00007fa2014d3000] java.lang.Thread.State: WAITING (on object monitor) at java.lang.Object.wait(Native Method) - waiting on <0x00000007b6801830> (a java.lang.ref.Reference$Lock) at java.lang.Object.wait(Object.java:503) at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:133) - locked <0x00000007b6801830> (a java.lang.ref.Reference$Lock) "main" prio=10 tid=0x00007fa20c00a000 nid=0x2183 runnable [0x00007fa2148a1000] java.lang.Thread.State: RUNNABLE at java.lang.ClassLoader.loadClass(ClassLoader.java:416) - locked <0x00000007ab4251e0> (a java.lang.Object) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:270) at sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:114) at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125) at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49) at sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:390) at sun.reflect.annotation.AnnotationParser.parseAnnotation(AnnotationParser.java:200) at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:88) at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:70) at java.lang.Class.initAnnotationsIfNecessary(Class.java:3178) - locked <0x00000007ab4239f8> (a java.lang.Class for org.apache.hadoop.ipc.metrics.RetryCacheMetrics) at java.lang.Class.getAnnotations(Class.java:3158) at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder.initRegistry(MetricsSourceBuilder.java:114) at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder.<init>(MetricsSourceBuilder.java:65) at org.apache.hadoop.metrics2.lib.MetricsAnnotations.newSourceBuilder(MetricsAnnotations.java:42) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:224) - locked <0x00000007b7ae9500> (a org.apache.hadoop.metrics2.impl.MetricsSystemImpl) at org.apache.hadoop.ipc.metrics.RetryCacheMetrics.create(RetryCacheMetrics.java:54) at org.apache.hadoop.ipc.RetryCache.<init>(RetryCache.java:202) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initRetryCache(FSNamesystem.java:906) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:827) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:689) at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:1000) at org.apache.hadoop.hdfs.server.namenode.NameNode.format(NameNode.java:361) at org.apache.hadoop.hdfs.DFSTestUtil.formatNameNode(DFSTestUtil.java:220) at org.apache.hadoop.hdfs.MiniDFSCluster.configureNameService(MiniDFSCluster.java:966) at org.apache.hadoop.hdfs.MiniDFSCluster.createNameNodesAndSetConf(MiniDFSCluster.java:887) at org.apache.hadoop.hdfs.MiniDFSCluster.initMiniDFSCluster(MiniDFSCluster.java:819) at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:478) at org.apache.hadoop.hdfs.MiniDFSCluster$Builder.build(MiniDFSCluster.java:437) at org.apache.hadoop.hdfs.web.TestWebHdfsTokens.testLazyTokenFetchForSWebhdfs(TestWebHdfsTokens.java:221) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124) at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103) "VM Thread" prio=10 tid=0x00007fa20c264800 nid=0x2191 runnable "GC task thread#0 (ParallelGC)" prio=10 tid=0x00007fa20c020000 nid=0x2184 runnable "GC task thread#1 (ParallelGC)" prio=10 tid=0x00007fa20c022000 nid=0x2185 runnable "GC task thread#2 (ParallelGC)" prio=10 tid=0x00007fa20c023800 nid=0x2186 runnable "GC task thread#3 (ParallelGC)" prio=10 tid=0x00007fa20c025800 nid=0x2187 runnable "GC task thread#4 (ParallelGC)" prio=10 tid=0x00007fa20c027800 nid=0x2188 runnable "GC task thread#5 (ParallelGC)" prio=10 tid=0x00007fa20c029800 nid=0x2189 runnable "GC task thread#6 (ParallelGC)" prio=10 tid=0x00007fa20c02b000 nid=0x218a runnable "GC task thread#7 (ParallelGC)" prio=10 tid=0x00007fa20c02d000 nid=0x218b runnable "GC task thread#8 (ParallelGC)" prio=10 tid=0x00007fa20c02f000 nid=0x218c runnable "GC task thread#9 (ParallelGC)" prio=10 tid=0x00007fa20c030800 nid=0x218d runnable "GC task thread#10 (ParallelGC)" prio=10 tid=0x00007fa20c032800 nid=0x218e runnable "GC task thread#11 (ParallelGC)" prio=10 tid=0x00007fa20c034800 nid=0x218f runnable "GC task thread#12 (ParallelGC)" prio=10 tid=0x00007fa20c036800 nid=0x2190 runnable "VM Periodic Task Thread" prio=10 tid=0x00007fa20c2a4800 nid=0x2198 waiting on condition JNI global references: 273 + echo '************ END zombies jstack extract' ************ END zombies jstack extract + JIRA_COMMENT=' {color:red}-1 core zombie tests{color}. There are 1 zombie test(s): at org.apache.hadoop.hdfs.web.TestWebHdfsTokens.testLazyTokenFetchForSWebhdfs(TestWebHdfsTokens.java:221)' + BAD=1 + jps + grep surefirebooter + cut -d ' ' -f 1 + xargs kill -9 POST BUILD TASK : SUCCESS END OF POST BUILD TASK : 0 Archiving artifacts Sending artifact delta relative to HBase-1.3-IT » IntegrationTestAcidGuarantees,latest1.8,Hadoop #182 Archived 2 artifacts Archive block size is 32768 Received 0 blocks and 6689738 bytes Compression is 0.0% Took 1.6 sec
