Hi Esteban, I'm using HBase 0.94.7. I have a cluster of 5 RS + 1 master. This is happening on the RS containing a region that is being accessed very frequently. I'm not seeing any particular log during that time.
Thanks, Pablo. 2013/7/30 Esteban Gutierrez <[email protected]> > Hi Pablo, > > > What do you see in the logs around the time you saw that behavior? is > this happening on a single Region Server? what version of HBase are > you running? > > cheers, > Esteban. > > Cloudera, Inc. > > On Jul 29, 2013, at 20:21, Pablo Medina <[email protected]> wrote: > > > Hi all, > > > > I'm having a lot of handlers (90 - 300 aprox) being blocked when reading > > rows. They are blocked during changedReaderObserver registration. > > Does anybody else run into the same issue? > > > > Stack trace: > > > > "IPC Server handler 99 on 60020" daemon prio=10 tid=0x0000000041c84000 > > nid=0x2244 waiting on condition [0x00007ff51fefd000] > > java.lang.Thread.State: WAITING (parking) > > at sun.misc.Unsafe.park(Native Method) > > - parking to wait for <0x00000000c5c13ae8> (a > > java.util.concurrent.locks.ReentrantLock$NonfairSync) > > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:156) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:811) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:842) > > at > > > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1178) > > at > > > java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:186) > > at > > java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:262) > > at > > > java.util.concurrent.CopyOnWriteArrayList.addIfAbsent(CopyOnWriteArrayList.java:553) > > at > > > java.util.concurrent.CopyOnWriteArraySet.add(CopyOnWriteArraySet.java:221) > > at > > > org.apache.hadoop.hbase.regionserver.Store.addChangedReaderObserver(Store.java:1085) > > at > > > org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:138) > > at > > org.apache.hadoop.hbase.regionserver.Store.getScanner(Store.java:2077) > > at > > > org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3755) > > at > > > org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1804) > > at > > > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1796) > > at > > > org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1771) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4776) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:4750) > > at > > > org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2152) > > at > > > org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3700) > > at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at > > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320) > > at > > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426) > > > > Thanks, > > Pablo. >
