Apologies for the long post but being an HBase newbie I'd like a sanity check
from wiser, more experienced folks with regards to my findings, and how I've
fixed the issue I think I've found, before going down the path of submitting
a formal patch.
To re-create the problem I was seeing in my app I added the following code
to TestIndexTable,
import org.apache.hadoop.hbase.client.RowLock;
private void assertRowUpdated(int updatedRow, int expectedRowValue)
throws IndexNotFoundException, IOException {
ResultScanner scanner = table.getIndexedScanner(INDEX_COL_A, null, null,
null, null, null);
byte[] persistedRowValue = null;
for (Result rowResult : scanner) {
byte[] row = rowResult.getRow();
byte[] value = rowResult.getValue(COL_A);
if
(Bytes.toString(row).equals(Bytes.toString(PerformanceEvaluation.format(updatedRow))))
{
persistedRowValue = value;
LOG.info("update found: row [" + Bytes.toString(row)
+ "] value [" + Bytes.toString(value) + "]");
}
else
LOG.info("updated index scan : row [" + Bytes.toString(row)
+ "] value [" + Bytes.toString(value) + "]");
}
scanner.close();
Assert.assertEquals(Bytes.toString(PerformanceEvaluation.format(expectedRowValue)),
Bytes.toString(persistedRowValue));
}
private void updateRow(int row, int newValue) throws IOException {
Put update = new Put(PerformanceEvaluation.format(row));
byte[] valueA = PerformanceEvaluation.format(newValue);
update.add(FAMILY, QUAL_A, valueA);
table.put(update);
LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val:
["
+ Bytes.toString(valueA) + "]");
}
private void updateLockedRow(int row, int newValue) throws IOException {
RowLock lock = table.lockRow(PerformanceEvaluation.format(row));
Put update = new Put(PerformanceEvaluation.format(row), lock);
byte[] valueA = PerformanceEvaluation.format(newValue);
update.add(FAMILY, QUAL_A, valueA);
LOG.info("Updating row [" + Bytes.toString(update.getRow()) + "] val:
["
+ Bytes.toString(valueA) + "]");
table.put(update);
LOG.info("Updated row [" + Bytes.toString(update.getRow()) + "] val:
["
+ Bytes.toString(valueA) + "]");
table.unlockRow(lock);
}
public void testRowUpdate() throws IOException {
writeInitalRows();
int row = NUM_ROWS - 2;
int value = MAX_VAL + 111;
updateRow(row, value);
assertRowUpdated(row, value);
}
public void testLockedRowUpdate() throws IOException {
writeInitalRows();
int row = NUM_ROWS - 2;
int value = MAX_VAL + 111;
updateLockedRow(row, value);
assertRowUpdated(row, value);
}
As I expected from the behavior of my own app the test testRowUpdate() was
successful but the test testLockedRowUpdate() had an error as follows,
[junit] java.io.IOException: java.io.IOException: Invalid row lock
[junit] at
org.apache.hadoop.hbase.regionserver.HRegion.getLock(HRegion.java:1640)
[junit] at
org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1244)
[junit] at
org.apache.hadoop.hbase.regionserver.tableindexed.IndexedRegion.put(IndexedRegion.java:97)
[junit] at
org.apache.hadoop.hbase.regionserver.HRegion.put(HRegion.java:1216)
[junit] at
org.apache.hadoop.hbase.regionserver.HRegionServer.put(HRegionServer.java:1818)
[junit] at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
[junit] at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[junit] at java.lang.reflect.Method.invoke(Method.java:597)
[junit] at
org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:650)
[junit] at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
NOTE: Line numbers in stacktrace may not make sense because I've been
hacking in loads of debug info.
To fix the error I made three changes to IndexedRegion.java. The changes are
on lines marked *KST* below and in summary all they do is propagate the
lockid.
public void put(Put put, Integer lockId, boolean writeToWAL)
throws IOException {
updateIndexes(put, lockId); // Do this first because will want to see
the old row *KST*
super.put(put, lockId, writeToWAL);
}
private void updateIndexes(Put put, Integer lockId) throws IOException {
// *KST*
List<IndexSpecification> indexesToUpdate = new
LinkedList<IndexSpecification>();
// Find the indexes we need to update
for (IndexSpecification index : getIndexes()) {
if (possiblyAppliesToIndex(index, put)) {
indexesToUpdate.add(index);
}
}
if (indexesToUpdate.size() == 0) {
return;
}
NavigableSet<byte[]> neededColumns =
getColumnsForIndexes(indexesToUpdate);
NavigableMap<byte[], byte[]> newColumnValues = getColumnsFromPut(put);
Get oldGet = new Get(put.getRow());
for (byte [] neededCol : neededColumns) {
oldGet.addColumn(neededCol);
}
Result oldResult = super.get(oldGet, lockId); // *KST*
Feedback as to how I should proceed with these findings would be great. e.g.
should I submit a formal patch, have I made a silly coding error in my test
or in IndexedRegion? Also, maybe the test are insufficient. I have al sorts
of variations I have tried with autoflush turned off, maybe I should include
those tests too?
--
View this message in context:
http://www.nabble.com/Row-lock-issues-with-Indexed-Tables-tp25442355p25442355.html
Sent from the HBase User mailing list archive at Nabble.com.
