Row locks don't move with regions or splits... are such things happening frequently?
Also, any reason to not use HBase's own increment method that's much more efficient? http://hbase.apache.org/apidocs/org/apache/hadoop/hbase/client/HTable.html#incrementColumnValue(byte[], byte[], byte[], long) J-D On Tue, Jul 24, 2012 at 2:53 PM, Marco Gallotta <[email protected]> wrote: > Hi there > > I am getting an UnknownRowLockException when adding locks to the increment() > function below. The full stack trace is at the end of this message. > > There are a few other places I am acquiring locks, but I only ever acquire a > single lock for one piece of code and the rest go through fine. It's only > when I enable the locks in this function that I get the exception. It's > fairly simple, and I can't find anything obviously wrong with it so my best > guess is there's some quirk with hbase locks that I'm unaware of. Any ideas? > > One thing to note is that I am calling increment() a couple times > consecutively for the same row. > > Marco > > public static void increment(String tableName, String key, String family, > String qualifier, int value, boolean useShort, > Configuration config) > throws IOException{ > HTable table = new HTable(config, > config.get("app", "geomon-test") + "." + tableName); > Get get = new Get(key.getBytes()); > get.addColumn(family.getBytes(), > qualifier.getBytes()); > int before = 0; > RowLock lock = table.lockRow(key.getBytes()); > try { > Result result = table.get(get); > if (!result.isEmpty()) { > if (useShort) { > before = (int) Bytes.toShort(result.getValue(family.getBytes(), > qualifier.getBytes())); > } else { > before = Bytes.toInt(result.getValue(family.getBytes(), > qualifier.getBytes())); > } > } > if (useShort) { > value = Math.min(before + value, Short.MAX_VALUE); > } else { > value = (int) Math.min((long) before + value, Integer.MAX_VALUE); > } > > Put put = new Put(key.getBytes()); > put.add(family.getBytes(), > qualifier.getBytes(), > Bytes.toBytes(useShort ? (short) value : value)); > table.put(put); > } catch (Exception e) { > System.err.println(e.getMessage()); > } finally { > table.unlockRow(lock); > } > } > > > 12/07/24 14:20:58 INFO mapred.JobClient: Task Id : > attempt_201207241320_0002_r_000000_2, Status : > FAILEDorg.apache.hadoop.hbase.UnknownRowLockException: > org.apache.hadoop.hbase.UnknownRowLockException: -3110061788478328763 > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:532) > at > org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:95) > at > org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79) > at > org.apache.hadoop.hbase.client.ServerCallable.translateException(ServerCallable.java:228) > at > org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:166) > at org.apache.hadoop.hbase.client.HTable.unlockRow(HTable.java:1031) > at Util.increment(Util.java:170) > at Aggregate$EventReducer.processUserEvent(Aggregate.java:161) > at Aggregate$EventReducer.processUser(Aggregate.java:119) > at Aggregate$EventReducer.reduce(Aggregate.java:189) > at Aggregate$EventReducer.reduce(Aggregate.java:78) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176) > at > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:417) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:416) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: org.apache.hadoop.ipc.RemoteException: > org.apache.hadoop.hbase.UnknownRowLockException: -3110061788478328763 > at > org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(HRegionServer.java:2633) > at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > at java.lang.reflect.Method.invoke(Method.java:616) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1376) > > at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:918) > at > org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:150) > at $Proxy3.unlockRow(Unknown Source) > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1033) > at org.apache.hadoop.hbase.client.HTable$15.call(HTable.java:1031) > at > org.apache.hadoop.hbase.client.ServerCallable.withRetries(ServerCallable.java:163) > ... 14 more > > > > -- > Marco Gallotta | Mountain View, California > Software Engineer, Infrastructure | Loki Studios > fb.me/marco.gallotta | twitter.com/marcog > [email protected] | +1 (650) 417-3313 > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) >
