I'm not on 0.94.1, but I've found a lot of situations that can cause scanner timeouts and other scanner exceptions from M/R. The primary ones probably still apply in later versions:
- Caching or batching set too high. If caching is set to, e.g. 1000, and hbase.rpc.timeout is set to 30 seconds, it means you need to be able to be able to process 1000 records in your mapper in under 30 seconds (minus the overhead of actually returning that many records). Otherwise the mapper's next call to next() will throw a timeout. - Similar to the above, this can happen if just the logic in your mapper is too heavy and taking too long. Just keep in mind that the hbase.rpc.timeout can be triggered by the time between calls to next() - hbase.rpc.timeout > hbase.regionserver.lease.period. If this is the case, the RS will timeout first and kill the scan. Then your mapper will call next() and since the scan no longer exists will throw a scan/lease exception. - The filters on the mapper scan are causing too many rows to be skipped, such that not enough rows can be collected to return within the timeout. Hope this helps and/or is still accurate for your version. On Mon, Oct 22, 2012 at 11:51 AM, J Mohamed Zahoor <[email protected]> wrote: > I am using 0.94.1 > > ./zahoor > > > On 22-Oct-2012, at 9:17 PM, J Mohamed Zahoor <[email protected]> wrote: > > > Hi > > > > I am facing a scanner exception like this when i run a mr job. > > Both the input and output are hbase tables (different tables)… > > This comes sporadically on some mapper and all other mappers runs fine.. > > Even the failed mapper gets passed in the next attempt. > > Any clue on what might be wrong? > > > > java.lang.NullPointerException > > at org.apache.hadoop.hbase.client.Scan.<init>(Scan.java:147) > > at > org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.createRecordReader(TableInputFormatBase.java:123) > > at > org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:489) > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731) > > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > > at java.security.AccessController.doPrivileged(Native Method) > > at javax.security.auth.Subject.doAs(Subject.java:415) > > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093) > > at org.apache.hadoop.mapred.Child.main(Child.java:249) > > > > > > ./zahoor > >
