I'm not sure if people have been using row locks with ITHBase. I havent been.
So looks like you found an issue, and have a solution for it. Sweet. Create a JIRA for it and attach your patch. cheers, -clint On Mon, Sep 14, 2009 at 12:52 PM, Keith Thomas <[email protected]>wrote: > > Apologies for the long post but being an HBase newbie I'd like a sanity > check > from wiser, more experienced folks with regards to my findings, and how > I've > fixed the issue I think I've found, before going down the path of > submitting > a formal patch. > > To re-create the problem I was seeing in my app I added the following code > to TestIndexTable, > > import org.apache.hadoop.hbase.client.RowLock; > > private void assertRowUpdated(int updatedRow, int expectedRowValue) > throws IndexNotFoundException, IOException { > ResultScanner scanner = table.getIndexedScanner(INDEX_COL_A, null, null, > null, null, null); > byte[] persistedRowValue = null; > for (Result rowResult : scanner) { > byte[] row = rowResult.getRow(); > byte[] value = rowResult.getValue(COL_A); > if > > (Bytes.toString(row).equals(Bytes.toString(PerformanceEvaluation.format(updatedRow)))) > { > persistedRowValue = value; > LOG.info("update found: row [" + Bytes.toString(row) > + "] value [" + Bytes.toString(value) + "]"); > } > else > LOG.info("updated index scan : row [" + Bytes.toString(row) > + "] value [" + Bytes.toString(value) + "]"); > } > scanner.close(); > > > > Assert.assertEquals(Bytes.toString(PerformanceEvaluation.format(expectedRowValue)), > Bytes.toString(persistedRowValue)); > } > > private void updateRow(int row, int newValue) throws IOException { > Put update = new Put(PerformanceEvaluation.format(row)); > byte[] valueA = PerformanceEvaluation.format(newValue); > update.add(FAMILY, QUAL_A, valueA); > table.put(update); > LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val: > [" > + Bytes.toString(valueA) + "]"); > } > > private void updateLockedRow(int row, int newValue) throws IOException { > RowLock lock = table.lockRow(PerformanceEvaluation.format(row)); > Put update = new Put(PerformanceEvaluation.format(row), lock); > byte[] valueA = PerformanceEvaluation.format(newValue); > update.add(FAMILY, QUAL_A, valueA); > LOG.info("Updating row [" + Bytes.toString(update.getRow()) + "] val: > [" > + Bytes.toString(valueA) + "]"); > table.put(update); > LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val: > [" > + Bytes.toString(valueA) + "]"); > table.unlockRow(lock); > } > > public void testRowUpdate() throws IOException { > writeInitalRows(); > int row = NUM_ROWS - 2; > int value = MAX_VAL + 111; > updateRow(row, value); > assertRowUpdated(row, value); > } > > public void testLockedRowUpdate() throws IOException { > writeInitalRows(); > int row = NUM_ROWS - 2; > int value = MAX_VAL + 111; > updateLockedRow(row, value); > assertRowUpdated(row, value); > } > > As I expected from the behavior of my own app the test testRowUpdate() was > successful but the test testLockedRowUpdate() had an error as follows, > > [junit] java.io.IOException: java.io.IOException: Invalid row lock > [junit] at > org.apache.hadoop.hbase.regionserver.HRegion.getLock(HRegion.java:1640) > [junit] at > org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1244) > [junit] at > > org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.put(IndexedRegion.java:97) > [junit] at > org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1216) > [junit] at > > org.apache.hadoop.hbase.regionserver.HRegionServer.put(HRegionServer.java:1818) > [junit] at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown > Source) > [junit] at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > [junit] at java.lang.reflect.Method.invoke(Method.java:597) > [junit] at > org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650) > [junit] at > org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915) > > NOTE: Line numbers in stacktrace may not make sense because I've been > hacking in loads of debug info. > > To fix the error I made three changes to IndexedRegion.java. The changes > are > on lines marked *KST* below and in summary all they do is propagate the > lockid. > > public void put(Put put, Integer lockId, boolean writeToWAL) > throws IOException { > updateIndexes(put, lockId); // Do this first because will want to see > the old row *KST* > super.put(put, lockId, writeToWAL); > } > > private void updateIndexes(Put put, Integer lockId) throws IOException { > // *KST* > List<IndexSpecification> indexesToUpdate = new > LinkedList<IndexSpecification>(); > > // Find the indexes we need to update > for (IndexSpecification index : getIndexes()) { > if (possiblyAppliesToIndex(index, put)) { > indexesToUpdate.add(index); > } > } > > if (indexesToUpdate.size() == 0) { > return; > } > > NavigableSet<byte[]> neededColumns = > getColumnsForIndexes(indexesToUpdate); > NavigableMap<byte[], byte[]> newColumnValues = getColumnsFromPut(put); > > Get oldGet = new Get(put.getRow()); > for (byte [] neededCol : neededColumns) { > oldGet.addColumn(neededCol); > } > > Result oldResult = super.get(oldGet, lockId); // *KST* > > Feedback as to how I should proceed with these findings would be great. > e.g. > should I submit a formal patch, have I made a silly coding error in my test > or in IndexedRegion? Also, maybe the test are insufficient. I have al sorts > of variations I have tried with autoflush turned off, maybe I should > include > those tests too? > -- > View this message in context: > http://www.nabble.com/Row-lock-issues-with-Indexed-Tables-tp25442355p25442355.html > Sent from the HBase User mailing list archive at Nabble.com. > >
