You may be running into HBASE-2481? St.Ack
On Tue, Apr 27, 2010 at 1:30 AM, steven zhuang <steven.zhuang.1...@gmail.com> wrote: > the first thread can be found at: > http://permalink.gmane.org/gmane.comp.java.hadoop.hbase.user/10074 > > After some dig, it seems that the problem is caused by long pause > between two "scanner.next()" call. > > In my case the program has to spent a relatively long while to > process one row, when it calls "scanner.next()" again, seems that the > returned Result will be null even if there should be more rows in the > tables. The rowcaching is set to 1. > I have checked some of the source code, seems there is some > mechanism which will call the close() method of the ClientScanner, but I am > still checking. > I don't know if there is a certain timeout on > ClientScanner/ScannerCallable after a row has been successfully returned, > seems that timeout cause my problem here. > > Any reply is appreciated. > > > On Fri, Apr 23, 2010 at 11:10 AM, steven zhuang < > steven.zhuang.1...@gmail.com> wrote: > >> hi, >> sorry I start another thread here. This mail is actually answer >> to a previous thread "multiple scanners on same table will cause problem? >> Scan results change among different tries.". >> the mail system kept saying that I am spamming, now it seems that >> it's right! :) >> >> here is my reply to people in that thread: >> >> I don't know if there is a limit on reads to a single row/region in >> HBase, but if there is, I might have exceeded that limit. :( >> in my case, there are hundreds of rows, with dozens of kilos of cells >> in a row(a 256 MB region may contain 10- rows). for each row, I started a >> thread on each CF, there are 8 of them, so there might be dozens of scanners >> on the same region. >> and, to Tim, I could not see your attached mail, my test code is >> pasted below, it just iterate on the rows and column families, output all >> the cells. >> >> private void doScan() throws Exception { >> if (null == CopyOfTestTTT234.table) { >> return; >> } >> Scan s = new Scan(); >> s.setStartRow("aaa".getBytes()); >> s.setStopRow("ccc".getBytes()); >> s.setCaching(CopyOfTestTTT234.ROWCACHING); //it's 1 here. >> ResultScanner scanner = CopyOfTestTTT234.table.getScanner(s); >> while (true) { >> Result row = scanner.next(); >> if(null==row) break; >> String rowKey = new String(row.getRow()); >> NavigableMap<byte[], NavigableMap<byte[], byte[]>> fm = row >> .getNoVersionMap(); >> while (fm.size() > 0) { >> Entry<byte[], NavigableMap<byte[], byte[]>> ee = fm >> .pollFirstEntry(); >> String fName = new String(ee.getKey()); >> NavigableMap<byte[], byte[]> ff = ee.getValue(); >> while (ff.size() > 0) { >> Entry<byte[], byte[]> cell = ff.pollFirstEntry(); >> String key = new String(cell.getKey()); >> String val = new String(cell.getValue()); >> System.out.println(Thread.currentThread().hashCode() + "\t" >> + rowKey + "\t" + fName + "\t" + key + "\t" + val); >> } >> } >> } >> } >> >
