""LRU Statistics #0" prio=10 tid=0x00007f4edc7dd800 nid=0x211a waitingon condition [0x00007f4e631e2000]" is not a daemon thread, but should be? - Andy
----- Original Message ----- > From: Roman Shaposhnik <[email protected]> > To: [email protected] > Cc: > Sent: Thursday, November 3, 2011 3:34 PM > Subject: HBase 0.92/Hadoop 0.22 test results > > So here's the run after I resolved all the set up issues: > > http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-trunk-smoketest-22/14/testReport/ > > Here's what I see timing out: > > http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-trunk-smoketest-22/14/testReport/org.apache.bigtop.itest.hbase.smoke/TestHBaseCompression/testNoCompression/ > > http://bigtop01.cloudera.org:8080/view/Hadoop%200.22/job/Bigtop-trunk-smoketest-22/14/testReport/org.apache.bigtop.itest.hbase.smoke/TestHBaseCompression/testGzipCompression/ > > Which is basically simply: > $ hbase org.apache.hadoop.hbase.util.CompressionTest > hdfs://ip-10-32-33-167.ec2.internal:17020/user/root/snappy-output/testfile.gz > gz > $ hbase org.apache.hadoop.hbase.util.CompressionTest > hdfs://ip-10-32-33-167.ec2.internal:17020/user/root/snappy-output/testfile.none > none > > And yes, they are hanging when executed by hand as well. Which is > weird too, since the > test itself actually completes, and exits > org.apache.hadoop.hbase.util.CompressionTest.main > and then everything freezes over with the following stack trace: > > $ jstack 8412 > 2011-11-03 18:33:37 > Full thread dump Java HotSpot(TM) 64-Bit Server VM (17.0-b16 mixed mode): > > "Attach Listener" daemon prio=10 tid=0x00007f4e6c001000 nid=0x213a > waiting on condition [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "DestroyJavaVM" prio=10 tid=0x00007f4edc009800 nid=0x2108 waiting on > condition [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "LeaseChecker" daemon prio=10 tid=0x00007f4edc7e6800 nid=0x211c > waiting on condition [0x00007f4e62fe0000] > java.lang.Thread.State: TIMED_WAITING (sleeping) > at java.lang.Thread.sleep(Native Method) > at org.apache.hadoop.hdfs.DFSClient$LeaseChecker.run(DFSClient.java:1476) > at java.lang.Thread.run(Thread.java:619) > > "LRU Statistics #0" prio=10 tid=0x00007f4edc7dd800 nid=0x211a waiting > on condition [0x00007f4e631e2000] > java.lang.Thread.State: TIMED_WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x00007f4e88acc968> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:198) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2025) > at java.util.concurrent.DelayQueue.take(DelayQueue.java:164) > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:583) > at > java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:576) > at > java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:947) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:907) > at java.lang.Thread.run(Thread.java:619) > > "LruBlockCache.EvictionThread" daemon prio=10 tid=0x00007f4edc7d8800 > nid=0x2119 in Object.wait() [0x00007f4e632e3000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x00007f4e88b1e570> (a > org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread) > at java.lang.Object.wait(Object.java:485) > at > org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread.run(LruBlockCache.java:568) > - locked <0x00007f4e88b1e570> (a > org.apache.hadoop.hbase.io.hfile.LruBlockCache$EvictionThread) > at java.lang.Thread.run(Thread.java:619) > > "Low Memory Detector" daemon prio=10 tid=0x00007f4edc0ff000 nid=0x2113 > runnable [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "CompilerThread1" daemon prio=10 tid=0x00007f4edc0fd000 nid=0x2112 > waiting on condition [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "CompilerThread0" daemon prio=10 tid=0x00007f4edc0fa800 nid=0x2111 > waiting on condition [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "Signal Dispatcher" daemon prio=10 tid=0x00007f4edc0f8800 nid=0x2110 > runnable [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "Surrogate Locker Thread (CMS)" daemon prio=10 tid=0x00007f4edc0f6800 > nid=0x210f waiting on condition [0x0000000000000000] > java.lang.Thread.State: RUNNABLE > > "Finalizer" daemon prio=10 tid=0x00007f4edc0d8800 nid=0x210e in > Object.wait() [0x00007f4ed031a000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x00007f4e896b0620> (a > java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118) > - locked <0x00007f4e896b0620> (a java.lang.ref.ReferenceQueue$Lock) > at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134) > at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159) > > "Reference Handler" daemon prio=10 tid=0x00007f4edc0d6800 nid=0x210d > in Object.wait() [0x00007f4ed041b000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x00007f4e896b0700> (a java.lang.ref.Reference$Lock) > at java.lang.Object.wait(Object.java:485) > at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116) > - locked <0x00007f4e896b0700> (a java.lang.ref.Reference$Lock) > > "VM Thread" prio=10 tid=0x00007f4edc0d2000 nid=0x210c runnable > > "Gang worker#0 (Parallel GC Threads)" prio=10 tid=0x00007f4edc018000 > nid=0x2109 runnable > > "Gang worker#1 (Parallel GC Threads)" prio=10 tid=0x00007f4edc019800 > nid=0x210a runnable > > "Concurrent Mark-Sweep GC Thread" prio=10 tid=0x00007f4edc078000 > nid=0x210b runnable > "VM Periodic Task Thread" prio=10 tid=0x00007f4edc10a000 nid=0x2114 > waiting on condition > > JNI global references: 1268 > > > Does this ring a bell, is this a problem we should pursue? > > Thanks, > Roman. >
