Deadlock while rolling WAL-log while finishing flush
----------------------------------------------------
Key: HBASE-709
URL: https://issues.apache.org/jira/browse/HBASE-709
Project: Hadoop HBase
Issue Type: Bug
Reporter: stack
Fix For: 0.1.3, 0.2.0
Found this in our cluster:
{code}
Found one Java-level deadlock:
=============================
"IPC Server handler 5 on 60020":
waiting for ownable synchronizer 0x00007f5f659d4e90, (a
java.util.concurrent.locks.ReentrantLock$NonfairSync),
which is held by "regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher"
"regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher":
waiting to lock monitor 0x00007f5f56312f78 (object 0x00007f5f659d4ec0, a
java.lang.Integer),
which is held by "IPC Server handler 5 on 60020"
Java stack information for the threads listed above:
===================================================
"IPC Server handler 5 on 60020":
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00007f5f659d4e90> (a
java.util.concurrent.locks.ReentrantLock$NonfairSync)
at java.util.concurrent.locks.LockSupport.park(Unknown Source)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(Unknown
Source)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(Unknown
Source)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(Unknown Source)
at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(Unknown
Source)
at java.util.concurrent.locks.ReentrantLock.lock(Unknown Source)
at org.apache.hadoop.hbase.HLog.rollWriter(HLog.java:219)
at org.apache.hadoop.hbase.HLog.append(HLog.java:390)
- locked <0x00007f5f659d4ec0> (a java.lang.Integer)
at org.apache.hadoop.hbase.HRegion.update(HRegion.java:1633)
at org.apache.hadoop.hbase.HRegion.batchUpdate(HRegion.java:1436)
at
org.apache.hadoop.hbase.HRegionServer.batchUpdate(HRegionServer.java:1552)
at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:413)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:901)
"regionserver/0:0:0:0:0:0:0:0:60020.cacheFlusher":
at org.apache.hadoop.hbase.HLog.completeCacheFlush(HLog.java:481)
- waiting to lock <0x00007f5f659d4ec0> (a java.lang.Integer)
at org.apache.hadoop.hbase.HRegion.internalFlushcache(HRegion.java:1133)
at org.apache.hadoop.hbase.HRegion.flushcache(HRegion.java:1020)
at
org.apache.hadoop.hbase.HRegionServer$Flusher.flushRegion(HRegionServer.java:446)
- locked <0x00007f5f659d3ec8> (a java.util.HashSet)
at
org.apache.hadoop.hbase.HRegionServer$Flusher.run(HRegionServer.java:390)
Found 1 deadlock.
{code}
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
