We're using hbase 0.90.0 here, and I'm seeing a curious behavior with my scans.
I have some code that does a scan over a table, and for each row
returned some work to verify the data...
I set the scan up like so :
byte[] family = Bytes.toBytes("mytable");
Scan scan = new Scan();
scan.setCaching(2000);
scan.addFamily(family);
and then scan using a fairly normal looking loop:
ResultScanner scanner = table.getScanner(scan);
for (Result userInfoResult : scanner) {
// do some work that takes about half a second
}
After this code runs for 60 seconds, I get the exception below:
Exception in thread "main" java.lang.RuntimeException:
org.apache.hadoop.hbase.client.ScannerTimeoutException: 78850ms passed
since the last invocation, timeout is currently set to 60000
at
org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1213)
at
cnwk.tridentp.firehose.ShowAssetMatchStatsMain.scanForProperties(ShowAssetMatchStatsMain.java:82)
at
cnwk.tridentp.firehose.ShowAssetMatchStatsMain.generateAssetMatchStats(ShowAssetMatchStatsMain.java:56)
at
cnwk.tridentp.firehose.ShowAssetMatchStatsMain.main(ShowAssetMatchStatsMain.java:33)
Caused by: org.apache.hadoop.hbase.client.ScannerTimeoutException:
78850ms passed since the last invocation, timeout is currently set to
60000
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1114)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner$1.hasNext(HTable.java:1210)
... 3 more
Caused by: org.apache.hadoop.hbase.UnknownScannerException:
org.apache.hadoop.hbase.UnknownScannerException: Name:
-3447615741604242626
at
org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1793)
at sun.reflect.GeneratedMethodAccessor31.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:570)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1036)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at
org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:83)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:38)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.java:999)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1100)
... 4 more
The message seems to indicate that I haven't called scanner.next() in
a long time (> 60 seconds), which isn't true, scanner.next() is being
called more than once per second.
However, if I disable caching (comment out the scan.setCaching() line
above), or set caching to a smaller value (1000, instead of 2000), the
code runs as expected.
I'm guessing that a cache-hit during my scan isn't resetting the
timeout that leads to a ScannerTimeoutException, but that's just a
guess...
Is this expected behavior?
...joe
