Well that error will be treated as fatal if it really took you more
than 60 seconds to do a next() invocation, but if you are using TIF
then the TableRecordReaderImpl will do the following:
try {
value = this.scanner.next();
} catch (IOException e) {
LOG.debug("recovered from " + StringUtils.stringifyException(e));
restart(lastRow);
scanner.next(); // skip presumed already mapped row
value = scanner.next();
}
So your mapper will recover. Do you see that message in your mappers' logs?
J-D
On Thu, Sep 16, 2010 at 9:57 AM, Ken Weiner <[email protected]> wrote:
> Hi J-D,
>
> Yes, I do see this INFO message on the same region server just a few seconds
> before the UnknownScannerException:
>
>
> 2010-09-16 09:20:49,953 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Scanner
> -8711007779313115048 lease expired
>
>
> I'm unclear about whether this is something I should try to address. Is
> this condition impacting the performance of the job and I should consider
> increasing the scanner lease time? If this is a completely normal behavior
> of HBase, is it reasonable to change the log level of that
> UnknownScannerException in the code from ERROR to WARN or INFO?
>
> Thanks.
>
> -Ken
>
> On Thu, Sep 16, 2010 at 9:38 AM, Jean-Daniel Cryans
> <[email protected]>wrote:
>
>> It's usually because your scanner timed out, or because the region
>> moved to a new server. You can see how it's handled in
>> HTable.ClientScanner.next(). In any case you should see a message like
>> "Scanner -8711007779313115048 lease expired" in some region server,
>> then see when you get the exception. Check if it's the same region
>> server, and the time between both.
>>
>> J-D
>>
>> On Thu, Sep 16, 2010 at 9:29 AM, Ken Weiner <[email protected]> wrote:
>> > Every time we run a map reduce job against data in HBase, we see
>> > hundreds of UnknownScannerExceptions in the hbase log at ERROR level.
>> > The job seems to complete fine and there are no other errors. Should
>> > I be concerned with these UnknownScannerExceptions? Is this message
>> > really more of a warning than an error?
>> >
>> >
>> >
>> > Example:
>> >
>> >
>> >
>> > 2010-09-16 09:20:52,398 ERROR
>> > org.apache.hadoop.hbase.regionserver.HRegionServer:
>> > org.apache.hadoop.hbase.UnknownScannerException: Name:
>> -8711007779313115048
>> > at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1880)
>> > at sun.reflect.GeneratedMethodAccessor8.invoke(Unknown Source)
>> > at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>> > at java.lang.reflect.Method.invoke(Method.java:597)
>> > at
>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:657)
>> > at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
>> >
>>
>
