I was unaware of locks not moving with regions/splits. Hmm…I just came across http://jerryjcw.blogspot.com/2009/10/hbase-notes-casual-remark-about-row.html which I'm going to try.
HBase's increment method says "readers do not take row locks so get and scan operations can see this operation partially completed", which could cause problems. Marco -- Marco Gallotta | Mountain View, California Software Engineer, Infrastructure | Loki Studios fb.me/marco.gallotta | twitter.com/marcog [email protected] | +1 (650) 417-3313 Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Tuesday 24 July 2012 at 2:58 PM, Jean-Daniel Cryans wrote: > Row locks don't move with regions or splits... are such things > happening frequently? > > Also, any reason to not use HBase's own increment method that's much > more efficient? > > http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#incrementColumnValue(byte[], > byte[], byte[], long) > > J-D > > On Tue, Jul 24, 2012 at 2:53 PM, Marco Gallotta <[email protected] > (mailto:[email protected])> wrote: > > Hi there > > > > I am getting an UnknownRowLockException when adding locks to the > > increment() function below. The full stack trace is at the end of this > > message. > > > > There are a few other places I am acquiring locks, but I only ever acquire > > a single lock for one piece of code and the rest go through fine. It's only > > when I enable the locks in this function that I get the exception. It's > > fairly simple, and I can't find anything obviously wrong with it so my best > > guess is there's some quirk with hbase locks that I'm unaware of. Any ideas? > > > > One thing to note is that I am calling increment() a couple times > > consecutively for the same row. > > > > Marco > > > > public static void increment(String tableName, String key, String family, > > String qualifier, int value, boolean useShort, > > Configuration config) > > throws IOException{ > > HTable table = new HTable(config, > > config.get("app", "geomon-test") + "." + tableName); > > Get get = new Get(key.getBytes()); > > get.addColumn(family.getBytes(), > > qualifier.getBytes()); > > int before = 0; > > RowLock lock = table.lockRow(key.getBytes()); > > try { > > Result result = table.get(get); > > if (!result.isEmpty()) { > > if (useShort) { > > before = (int) Bytes.toShort(result.getValue(family.getBytes(), > > qualifier.getBytes())); > > } else { > > before = Bytes.toInt(result.getValue(family.getBytes(), > > qualifier.getBytes())); > > } > > } > > if (useShort) { > > value = Math.min(before + value, Short.MAX_VALUE); > > } else { > > value = (int) Math.min((long) before + value, Integer.MAX_VALUE); > > } > > > > Put put = new Put(key.getBytes()); > > put.add(family.getBytes(), > > qualifier.getBytes(), > > Bytes.toBytes(useShort ? (short) value : value)); > > table.put(put); > > } catch (Exception e) { > > System.err.println(e.getMessage()); > > } finally { > > table.unlockRow(lock); > > } > > } > > > > > > 12/07/24 14:20:58 INFO mapred.JobClient: Task Id : > > attempt_201207241320_0002_r_000000_2, Status : > > FAILEDorg.apache.hadoop.hbase.UnknownRowLockException: > > org.apache.hadoop.hbase.UnknownRowLockException: -3110061788478328763 > > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > > at > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > > at > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > at java.lang.reflect.Constructor.newInstance(Constructor.java:532) > > at > > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95) > > at > > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79) > > at > > org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:228) > > at > > org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:166) > > at org.apache.hadoop.hbase.client.HTable.unlockRow(HTable.java:1031) > > at Util.increment(Util.java:170) > > at Aggregate$EventReducer.processUserEvent(Aggregate.java:161) > > at Aggregate$EventReducer.processUser(Aggregate.java:119) > > at Aggregate$EventReducer.reduce(Aggregate.java:189) > > at Aggregate$EventReducer.reduce(Aggregate.java:78) > > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) > > at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:416) > > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > Caused by: org.apache.hadoop.ipc.RemoteException: > > org.apache.hadoop.hbase.UnknownRowLockException: -3110061788478328763 > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(HRegionServer.java:2633) > > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > > at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > > at java.lang.reflect.Method.invoke(Method.java:616) > > at > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) > > at > > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) > > > > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:918) > > at > > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) > > at $Proxy3.unlockRow(Unknown Source) > > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1033) > > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1031) > > at > > org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:163) > > ... 14 more > > > > > > > > -- > > Marco Gallotta | Mountain View, California > > Software Engineer, Infrastructure | Loki Studios > > fb.me/marco.gallotta (http://fb.me/marco.gallotta) | twitter.com/marcog > > (http://twitter.com/marcog) > > [email protected] (mailto:[email protected]) | +1 (650) 417-3313 > > > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig)
