Hi Jim,
Thanks for the explanation, it's very helpful indeed =)
So if i understand correctly, whenever we open a scanner to perform
some batch operations on the table, there should not be any other
updates to the same table.
I suppose than it is also possible to manually catch the
UnknownScannerException in the middle of a long batch operation and
resume the operation by opening a new scanner starting from the row
key that just failed.
Do you think it is possible to handle this transparently in the
scanner.next() operation ?
Cheers,
Cedric
On 10/30/07, Jim Kellerman <[EMAIL PROTECTED]> wrote:
> UnknownScannerException is thrown by the region server if a scanner request
> (next or close) is called with either a bogus scanner id (unlikely since the
> scanner id is hidden from the client application) or if the scanner's lease
> has expired.
>
> Every client request is given a lease by the region server. If the lease
> times out, the region server thinks that the client has gone away and it can
> clean up any resources being held for that request. Lease timeouts almost
> always occur on operations related to scanners. If a scanner's lease times
> out, UnknownScannerException will be thrown if the client issues a scanner
> request after the timeout because the region server has already cleaned up
> the resources associated with the scanner. (The default lease timeout for
> client requests is 30 seconds, so it is very likely that this is the
> situation you are running into).
>
> If you attempt to do an update to the same table that you have a scanner open
> on, the update will stall waiting for the scanner to complete. So a loop
> which does
>
> while (scanner.next(key, val)) {
> if (key.equals(something) || val.equals(somethingElse)) {
> table.startUpdate(...)
> table.put(...)
> table.commit(...)
> }
> }
>
> will not work because the update will wait for the scanner to finish, (which
> it won't), and both the update and the next scanner.next should both fail.
>
> Hope that helps.
> ---
> Jim Kellerman, Senior Engineer; Powerset
> [EMAIL PROTECTED]
>
>
> > -----Original Message-----
> > From: Cedric Ho [mailto:[EMAIL PROTECTED]
> > Sent: Thursday, October 25, 2007 7:58 PM
> > To: [email protected]
> > Subject: HBase UnknownScannerException
> >
> > Hi,
> >
> > I was trying HBase from the 0.15 branch. And was doing:
> >
> > HScannerInterface s = table.obtainScanner(...)
> > while(s.next(key, val)) {
> > ....
> > }
> >
> > And encounter the following exception whenever I spent too
> > much time in the while loop, (e.g. > 30 ~ 120 seconds). I am
> > also doing insert in the same table in another process. But
> > the records they are working on are definitely non-overlapping.
> >
> > org.apache.hadoop.ipc.RemoteException:
> > org.apache.hadoop.hbase.UnknownScannerException: Name:
> > -7220522571873774180
> > at
> > org.apache.hadoop.hbase.HRegionServer.next(HRegionServer.java:1075)
> > at
> > sun.reflect.GeneratedMethodAccessor19.invoke(Unknown Source)
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMeth
> > odAccessorImpl.java:25)
> > at java.lang.reflect.Method.invoke(Method.java:585)
> > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:379)
> > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:596)
> >
> > at org.apache.hadoop.ipc.Client.call(Client.java:482)
> > at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184)
> > at $Proxy1.next(Unknown Source)
> > at
> > org.apache.hadoop.hbase.HTable$ClientScanner.next(HTable.java:831)
> > ...
> >
> > there are totally 6 machines:
> > one running namenode, secondarynamenode, hbase master five
> > running datanode, regionserver
> >
> >
> > Cheers,
> > Cedric
> >
>
