[
https://issues.apache.org/jira/browse/HBASE-19986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16361585#comment-16361585
]
stack commented on HBASE-19986:
-------------------------------
.003 does the @appy suggestion.... Here is what it looks like when it trips:
{code}
2018-02-12 15:08:31,261 INFO [Time-limited test] hbase.ResourceChecker(148):
before: TestTimeout#run1 Thread=8, OpenFileDescriptor=89,
MaxFileDescriptor=10240, SystemLoadAverage=478, ProcessCount=372,
AvailableMemoryMB=327
2018-02-12 15:08:31,444 INFO [Time-limited test] hbase.ResourceChecker(172):
after: TestTimeout#run1 Thread=8 (was 8), OpenFileDescriptor=89 (was 89),
MaxFileDescriptor=10240 (was 10240), SystemLoadAverage=478 (was 478),
ProcessCount=372 (was 372), AvailableMemoryMB=327 (was 327)
2018-02-12 15:08:31,523 INFO [Time-limited test] hbase.ResourceChecker(148):
before: TestTimeout#infiniteLoop Thread=8, OpenFileDescriptor=89,
MaxFileDescriptor=10240, SystemLoadAverage=478, ProcessCount=372,
AvailableMemoryMB=325
====> TEST TIMED OUT. PRINTING THREAD DUMP. <====
Timestamp: 2018-02-12 03:08:41,125
"surefire-forkedjvm-command-thread" daemon prio=5 tid=10 runnable
java.lang.Thread.State: RUNNABLE
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:255)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at java.io.DataInputStream.readInt(DataInputStream.java:387)
at
org.apache.maven.surefire.booter.MasterProcessCommand.decode(MasterProcessCommand.java:115)
at
org.apache.maven.surefire.booter.CommandReader$CommandRunnable.run(CommandReader.java:391)
at java.lang.Thread.run(Thread.java:745)
"Reference Handler" daemon prio=10 tid=2 in Object.wait()
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at java.lang.ref.Reference.tryHandlePending(Reference.java:191)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:153)
"main" prio=5 tid=1 runnable
java.lang.Thread.State: RUNNABLE
....
{code}
> If HBaseTestClassRule timesout a test, thread dump.
> ---------------------------------------------------
>
> Key: HBASE-19986
> URL: https://issues.apache.org/jira/browse/HBASE-19986
> Project: HBase
> Issue Type: Bug
> Reporter: stack
> Assignee: stack
> Priority: Major
> Attachments: HBASE-19986.branch-2.001.patch,
> HBASE-19986.branch-2.002.patch, HBASE-19986.branch-2.003.patch
>
>
> We set look for stuck thread in our timeout rule but it is super conservative
> in what it prints.. it looks for a RUNNABLE thread and prints first found
> ONLY. Pretty useless for us. If a test timesout, often the printing has
> stopped in the stderr/stdout.
> I'm trying to debug TestAsyncRegionAdminApi. It says test timed out after 10
> minutes but we've stopped printing to the logs and here is what junit prints:
> -------------------------------------------------------------------------------
> Test set: org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi
> -------------------------------------------------------------------------------
> Tests run: 25, Failures: 0, Errors: 2, Skipped: 2, Time elapsed: 572.508 s
> <<< FAILURE! - in org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi
> org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi Time elapsed: 14.642
> s <<< ERROR!
> org.junit.runners.model.TestTimedOutException: test timed out after 600
> seconds
> at
> org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi.testMergeRegions(TestAsyncRegionAdminApi.java:363)
> org.apache.hadoop.hbase.client.TestAsyncRegionAdminApi Time elapsed: 14.642
> s <<< ERROR!
> java.lang.Exception: Appears to be stuck in thread Socket Reader #1 for port
> 35917
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
