[ https://issues.apache.org/jira/browse/HBASE-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16329832#comment-16329832 ]
Duo Zhang commented on HBASE-19803: ----------------------------------- It seems a surefire issue. I run mvn test locally in hbase-server module, and TestJMXConnectorServer fails, this is a known issue, and then comes lots of crashes. This is one of the failed UT {noformat} Error occurred in starting fork, check output in log Process Exit Code: 1 Crashed tests: org.apache.hadoop.hbase.master.balancer.TestRegionsOnMasterOptions at org.apache.maven.plugin.surefire.booterclient.ForkStarter.awaitResultsDone(ForkStarter.java:496) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.runSuitesForkPerTestSet(ForkStarter.java:443) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:295) at org.apache.maven.plugin.surefire.booterclient.ForkStarter.run(ForkStarter.java:246) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:1124) at org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:954) ... 23 more Caused by: org.apache.maven.surefire.booter.SurefireBooterForkException: The forked VM terminated without properly saying goodbye. VM crash or System.exit called? Command was /bin/sh -c cd /home/zhangduo/hbase/code/hbase-server && /home/zhangduo/opt/jdk1.8.0_151/jre/bin/java -enableassertions -Dhbase.build.id=2018-01-17T22:44:23Z -Xmx2800m -Djava.security.egd=file:/dev/./urandom -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -jar /home/zhangduo/hbase/code/hbase-server/target/surefire/surefirebooter3125641250160453662.jar /home/zhangduo/hbase/code/hbase-server/target/surefire 2018-01-18T06-44-36_642-jvmRun2 surefire7506668156192398602tmp surefire_14263036952065448117423tmp {noformat} And I checked org.apache.hadoop.hbase.master.balancer.TestRegionsOnMasterOptions-output.txt, the only place where we call System.exit is {noformat} org.apache.hadoop.hbase.HConstants$ExitException: There is no escape! at org.apache.hadoop.hbase.HConstants$NoExitSecurityManager.checkExit(HConstants.java:63) at java.lang.Runtime.halt(Runtime.java:273) at org.apache.maven.surefire.booter.ForkedBooter.kill(ForkedBooter.java:300) at org.apache.maven.surefire.booter.ForkedBooter.kill(ForkedBooter.java:294) at org.apache.maven.surefire.booter.ForkedBooter.access$300(ForkedBooter.java:68) at org.apache.maven.surefire.booter.ForkedBooter$4.update(ForkedBooter.java:247) at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.insertToListeners(CommandReader.java:475) at org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:421) at java.lang.Thread.run(Thread.java:748) {noformat} Notice here I only log the exception without throwing it out if it is called from the surefire plugin. So it is killed by the surefire plugin? And then surefire plugin tells us the VM exited abnormally... > False positive for the HBASE-Find-Flaky-Tests job > ------------------------------------------------- > > Key: HBASE-19803 > URL: https://issues.apache.org/jira/browse/HBASE-19803 > Project: HBase > Issue Type: Bug > Reporter: Duo Zhang > Priority: Major > > It reports two hangs for TestAsyncTableGetMultiThreaded, but I checked the > surefire output > https://builds.apache.org/job/HBASE-Flaky-Tests/24830/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was likely to be killed in the middle of the run within 20 seconds. > https://builds.apache.org/job/HBASE-Flaky-Tests/24852/artifact/hbase-server/target/surefire-reports/org.apache.hadoop.hbase.client.TestAsyncTableGetMultiThreaded-output.txt > This one was also killed within about 1 minutes. > The test is declared as LargeTests so the time limit should be 10 minutes. It > seems that the jvm may crash during the mvn test run and then we will kill > all the running tests and then we may mark some of them as hang which leads > to the false positive. -- This message was sent by Atlassian JIRA (v7.6.3#76005)