Hi,
We use the following code to retrieve data from hbase 0.20.1 which didn't
return after 30 minutes:
ResultScanner _scanner = _data;
try {
Result[] _results = _scanner.next(defaultPageSize);
updateResults(pTable, _scanner, _results);
There is no exception thrown.
Here is stack traces of the client:
"IPC Client (47) connection to /10.10.31.135:60020 from an unknown user"
daemon prio=10 tid=0x00000000560de800 nid=0x34bc runnable
[0x000000004249b000]
java.lang.Thread.State: RUNNABLE
at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:215)
at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
- locked <0x00002aaaaffc9cb8> (a sun.nio.ch.Util$1)
- locked <0x00002aaaaffc9ca0> (a
java.util.Collections$UnmodifiableSet)
- locked <0x00002aaaaffc9678> (a sun.nio.ch.EPollSelectorImpl)
at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
at
org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:332)
at
org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157)
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
at
org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
at java.io.FilterInputStream.read(FilterInputStream.java:116)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:279)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
- locked <0x00002aaaaffb2878> (a java.io.BufferedInputStream)
at java.io.DataInputStream.readInt(DataInputStream.java:370)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:504)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:448)
"IPC Client (47) connection to /10.10.31.136:60020 from an unknown user"
daemon prio=10 tid=0x00000000565c1800 nid=0x34bb in Object.wait()
[0x000000004239a000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.waitForWork(HBaseClient.java:404)
- locked <0x00002aaaaffad298> (a
org.apache.hadoop.hbase.ipc.HBaseClient$Connection)
at
org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:447)
"AWT-EventQueue-0" prio=10 tid=0x0000000056456800 nid=0x34a8 in
Object.wait() [0x0000000041793000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:485)
at
org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:717)
- locked <0x00002aaaae2d8d48> (a
org.apache.hadoop.hbase.ipc.HBaseClient$Call)
at
org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:328)
at $Proxy0.close(Unknown Source)
at
org.apache.hadoop.hbase.client.ScannerCallable.close(ScannerCallable.java:101)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:72)
at
org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:38)
at
org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1887)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:2033)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:2051)
at
net.kindsight.webmap.gui.HBaseViewerPanel.updateUI(HBaseViewerPanel.java:390)
at
net.kindsight.webmap.gui.HBaseViewerPanel.access$900(HBaseViewerPanel.java:45)
at
net.kindsight.webmap.gui.HBaseViewerPanel$5.run(HBaseViewerPanel.java:362)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:199)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:597)
at
java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at
java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at
java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at
java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
The regionserver on 10.10.31.136 ran fine.
For regionserver on 10.10.31.135, apart from some WREs, things were normal:
2010-03-08 10:21:16,791 ERROR [IPC Server handler 17 on 60020]
regionserver.HRegionServer(844):
org.apache.hadoop.hbase.regionserver.WrongRegionException: Requested row out
of range for HRegion
ruletable,com.hoovers.www\x2Fcompanyindex\x2FMichigan\x2FBattle_Creek\x2FMarketing_and_Advertising_Services-1.html,1267860376861,
startKey='com.hoovers.www\x2Fcompanyindex\x2FMichigan\x2FBattle_Creek\x2FMarketing_and_Advertising_Services-1.html',
getEndKey()='com.hoovers.www\x2Fcompanyindex\x2FNew_Mexico\x2FVado\x2FBroadcasting_Industry-1.html',
row='com.xmradio.www\x2FpadData\x2Fpad_data_servlet.jsp\x3Fchannel\x3D66\x26rpc\x3DXMROUS\x26rnd\x3D8378'
at org.apache.hadoop.hbase.regionserver.HRegion.checkRow(HRegion.java:1522)
at
org.apache.hadoop.hbase.regionserver.HRegion.obtainRowLock(HRegion.java:1554)
at org.apache.hadoop.hbase.regionserver.HRegion.getLock(HRegion.java:1622)
at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:2278)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1785)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:648)
at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:915)
Has someone seen similar situation ?
Thanks
