Anoop attached backported patch in HBASE-8655 It should go into 0.94.9, the next release - current is 0.94.8
Cheers On Thu, May 30, 2013 at 7:01 AM, Kireet <[email protected]> wrote: > > > How long do backports typically take? We have to go live in a month ready > or not. Thanks for the quick replies Anoop and Ted. > > --Kireet > > > On 5/30/13 9:20 AM, Ted Yu wrote: > >> 0.95 client is not compatible with 0.94 cluster. So you cannot use 0.95 >> client. >> >> Cheers >> >> On May 30, 2013, at 6:12 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+** >> [email protected] <kireet-teh5dpvpl8nqt0dzr%[email protected]>> >> wrote: >> >> >>> >>> Would there be a problem if our cluster is 0.94 and we use a 0.95 client? >>> >>> I am not familiar with the HBase code base, but I did a dump of the >>> thread that is actually running (below). It seems like it is related to the >>> issue you mentioned as the running thread is doing the prefetch logic. >>> Would pre-splitting tables help here? We are doing some performance tests >>> and essentially starting from an empty instance. >>> >>> java.lang.Thread.State: WAITING (on object monitor) >>> at java.lang.Object.wait(Native Method) >>> at java.lang.Object.wait(Object.**java:503) >>> at org.apache.zookeeper.**ClientCnxn.submitRequest(** >>> ClientCnxn.java:1309) >>> - locked <0x00000000e10cf830> (a org.apache.zookeeper.** >>> ClientCnxn$Packet) >>> at org.apache.zookeeper.**ZooKeeper.exists(ZooKeeper.**java:1036) >>> at org.apache.hadoop.hbase.**zookeeper.**RecoverableZooKeeper.exists(** >>> RecoverableZooKeeper.java:172) >>> at org.apache.hadoop.hbase.**zookeeper.ZKUtil.checkExists(** >>> ZKUtil.java:450) >>> at org.apache.hadoop.hbase.**zookeeper.**ZooKeeperNodeTracker.** >>> checkIfBaseNodeAvailable(**ZooKeeperNodeTracker.java:208) >>> at org.apache.hadoop.hbase.**zookeeper.RootRegionTracker.** >>> waitRootRegionLocation(**RootRegionTracker.java:77) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegion(**HConnectionManager.java:874) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegionInMeta(** >>> HConnectionManager.java:987) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegion(**HConnectionManager.java:885) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegion(**HConnectionManager.java:846) >>> at org.apache.hadoop.hbase.**client.HTable.finishSetup(** >>> HTable.java:234) >>> at org.apache.hadoop.hbase.**client.HTable.<init>(HTable.**java:174) >>> at org.apache.hadoop.hbase.**client.MetaScanner.metaScan(** >>> MetaScanner.java:160) >>> at org.apache.hadoop.hbase.**client.MetaScanner.access$000(** >>> MetaScanner.java:54) >>> at org.apache.hadoop.hbase.**client.MetaScanner$1.connect(** >>> MetaScanner.java:133) >>> at org.apache.hadoop.hbase.**client.MetaScanner$1.connect(** >>> MetaScanner.java:130) >>> at org.apache.hadoop.hbase.**client.HConnectionManager.** >>> execute(HConnectionManager.**java:383) >>> at org.apache.hadoop.hbase.**client.MetaScanner.metaScan(** >>> MetaScanner.java:130) >>> at org.apache.hadoop.hbase.**client.MetaScanner.metaScan(** >>> MetaScanner.java:105) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**prefetchRegionCache(** >>> HConnectionManager.java:947) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegionInMeta(** >>> HConnectionManager.java:1002) >>> - locked <0x00000000e882f778> (a java.lang.Object) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegion(**HConnectionManager.java:889) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**locateRegion(**HConnectionManager.java:846) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**processBatchCallback(** >>> HConnectionManager.java:1533) >>> at org.apache.hadoop.hbase.**client.HConnectionManager$** >>> HConnectionImplementation.**processBatch(**HConnectionManager.java:1418) >>> at org.apache.hadoop.hbase.**client.HTable.batch(HTable.**java:702) >>> at org.apache.hadoop.hbase.**client.HTable.get(HTable.java:**671) >>> at org.apache.hadoop.hbase.**client.HTablePool$** >>> PooledHTable.get(HTablePool.**java:367) >>> >>> Thanks >>> Kireet >>> >>> On 5/30/13 12:16 AM, Anoop John wrote: >>> >>>> Can you have a look at issue HBASE-8476? Seems related? A fix is >>>> available in HBASE-8346's patch.. >>>> >>>> -Anoop- >>>> >>>> On Thu, May 30, 2013 at 9:21 AM, Kireet <kireet-Teh5dPVPL8nQT0dZR+** >>>> [email protected]<kireet-teh5dpvpl8nqt0dzr%[email protected]>> >>>> wrote: >>>> >>>> We are running hbase 0.94.6 in a concurrent environment and we are >>>>> seeing >>>>> the majority of our code stuck in this method at the synchronized block >>>>> (line 997). Is there some configuration we can change to avoid it? It >>>>> seems >>>>> like we could disable prefetching, but that sounds like it could cause >>>>> other problems? >>>>> >>>>> - org.apache.hadoop.hbase.****client.HConnectionManager$** >>>>> HConnectionImplementation.****locateRegionInMeta(byte[], byte[], >>>>> byte[], >>>>> boolean, java.lang.Object, boolean) @bci=149, line=997 (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HConnectionManager$** >>>>> HConnectionImplementation.****locateRegion(byte[], byte[], boolean, >>>>> boolean) @bci=212, line=889 (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HConnectionManager$** >>>>> HConnectionImplementation.****locateRegion(byte[], byte[]) @bci=5, >>>>> line=846 >>>>> (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HConnectionManager$** >>>>> HConnectionImplementation.****processBatchCallback(java.****util.List, >>>>> byte[], java.util.concurrent.****ExecutorService, java.lang.Object[], >>>>> org.apache.hadoop.hbase.****client.coprocessor.Batch$****Callback) >>>>> @bci=194, >>>>> line=1533 (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HConnectionManager$** >>>>> HConnectionImplementation.****processBatch(java.util.List, byte[], >>>>> java.util.concurrent.****ExecutorService, java.lang.Object[]) @bci=30, >>>>> line=1418 (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HTable.batch(java.util.** >>>>> **List) >>>>> @bci=24, line=702 (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HTable.get(java.util.****List) >>>>> @bci=2, >>>>> line=671 (Compiled frame) >>>>> - org.apache.hadoop.hbase.****client.HTablePool$** >>>>> PooledHTable.get(java.util.****List) @bci=5, line=367 (Compiled frame) >>>>> >>>> >>> >> >
