What version of ZK is being used? See: https://issues.apache.org/jira/browse/ZOOKEEPER-846
Patrick On Mon, Apr 4, 2011 at 10:48 AM, Stack <[email protected]> wrote: > Have you lot ever seen a hang on client close? A user over in > hbaselandia is seeing such a thing reportedly (See below -- I'm trying > to get more info). I thought I'd come ask the experts for their take. > Should we be running close of zk connection inside a timer > interrupting it if it gets stuck? > > Good stuff, > St.Ack > > > If I dump the JVM threads in a console, it looks like the region > server wants to close all zookeeper connections and blocks until this > is done. > HRegionServer.java:672 --> HConnectionManager.deleteConnection(conf, true); > > "regionserver60020-EventThread" daemon prio=10 tid=0x000000005d0dc000 > nid=0x7e1d waiting on condition [0x0000000042941000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0000000781c9ce00> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) > > "regionserver60020" prio=10 tid=0x00002aaab023e000 nid=0x7e1b in > Object.wait() [0x000000004273f000] > java.lang.Thread.State: WAITING (on object monitor) > at java.lang.Object.wait(Native Method) > - waiting on <0x00000007be9ad330> (a > org.apache.zookeeper.ClientCnxn$Packet) > at java.lang.Object.wait(Object.java:485) > at org.apache.zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1317) > - locked <0x00000007be9ad330> (a org.apache.zookeeper.ClientCnxn$Packet) > at org.apache.zookeeper.ClientCnxn.close(ClientCnxn.java:1295) > at org.apache.zookeeper.ZooKeeper.close(ZooKeeper.java:531) > - locked <0x0000000781fae170> (a org.apache.zookeeper.ZooKeeper) > at > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.close(ZooKeeperWatcher.java:399) > at > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.close(HConnectionManager.java:1050) > at > org.apache.hadoop.hbase.client.HConnectionManager.deleteConnection(HConnectionManager.java:175) > - locked <0x00000007801b6800> (a > org.apache.hadoop.hbase.client.HConnectionManager$1) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:672) > at java.lang.Thread.run(Thread.java:662) > > "main-EventThread" daemon prio=10 tid=0x00002aaab0540000 nid=0x7e10 > waiting on condition [0x0000000042139000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x0000000781faf3e0> (a > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:158) > at > java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1987) > at > java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399) > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) > > Unfortunately, some connections are never closed so the server does > not shut down. >
