thank you guys, I think Stack is right, what the ticket described is exactly what I met, the scanner close silently, there is no exception thrown out. And hi, Tim, I think you can reproduce my case by sleep 1 or 2 minutes between two next() calls. :)
Again, thanks you guys. Looking forward to release 0.20.4. :D On Tue, Apr 27, 2010 at 9:39 PM, Tim Robertson <timrobertson...@gmail.com>wrote: > Hi, > > I missed that comment about not being able to see my test code. Sorry > about > that, and here it is: https://dl.dropbox.com/u/608155/HBase/Test.java > > It's just a quick hack and nothing clever, but has 10000 threads (500 > running concurrently) running the scan and I have repeated it many times > with no errors (it is only 10 rows, but easy to modify). > > Cheers, > Tim > > > > > > > On Tue, Apr 27, 2010 at 3:31 PM, Stack <st...@duboce.net> wrote: > > > You may be running into HBASE-2481? > > St.Ack > > > > On Tue, Apr 27, 2010 at 1:30 AM, steven zhuang > > <steven.zhuang.1...@gmail.com> wrote: > > > the first thread can be found at: > > > http://permalink.gmane.org/gmane.comp.java.hadoop.hbase.user/10074 > > > > > > After some dig, it seems that the problem is caused by long > > pause > > > between two "scanner.next()" call. > > > > > > In my case the program has to spent a relatively long while > to > > > process one row, when it calls "scanner.next()" again, seems that the > > > returned Result will be null even if there should be more rows in the > > > tables. The rowcaching is set to 1. > > > I have checked some of the source code, seems there is some > > > mechanism which will call the close() method of the ClientScanner, but > I > > am > > > still checking. > > > I don't know if there is a certain timeout on > > > ClientScanner/ScannerCallable after a row has been successfully > returned, > > > seems that timeout cause my problem here. > > > > > > Any reply is appreciated. > > > > > > > > > On Fri, Apr 23, 2010 at 11:10 AM, steven zhuang < > > > steven.zhuang.1...@gmail.com> wrote: > > > > > >> hi, > > >> sorry I start another thread here. This mail is actually > > answer > > >> to a previous thread "multiple scanners on same table will cause > > problem? > > >> Scan results change among different tries.". > > >> the mail system kept saying that I am spamming, now it seems > > that > > >> it's right! :) > > >> > > >> here is my reply to people in that thread: > > >> > > >> I don't know if there is a limit on reads to a single row/region > > in > > >> HBase, but if there is, I might have exceeded that limit. :( > > >> in my case, there are hundreds of rows, with dozens of kilos of > > cells > > >> in a row(a 256 MB region may contain 10- rows). for each row, I > started > > a > > >> thread on each CF, there are 8 of them, so there might be dozens of > > scanners > > >> on the same region. > > >> and, to Tim, I could not see your attached mail, my test code is > > >> pasted below, it just iterate on the rows and column families, output > > all > > >> the cells. > > >> > > >> private void doScan() throws Exception { > > >> if (null == CopyOfTestTTT234.table) { > > >> return; > > >> } > > >> Scan s = new Scan(); > > >> s.setStartRow("aaa".getBytes()); > > >> s.setStopRow("ccc".getBytes()); > > >> s.setCaching(CopyOfTestTTT234.ROWCACHING); //it's 1 here. > > >> ResultScanner scanner = CopyOfTestTTT234.table.getScanner(s); > > >> while (true) { > > >> Result row = scanner.next(); > > >> if(null==row) break; > > >> String rowKey = new String(row.getRow()); > > >> NavigableMap<byte[], NavigableMap<byte[], byte[]>> fm = row > > >> .getNoVersionMap(); > > >> while (fm.size() > 0) { > > >> Entry<byte[], NavigableMap<byte[], byte[]>> ee = fm > > >> .pollFirstEntry(); > > >> String fName = new String(ee.getKey()); > > >> NavigableMap<byte[], byte[]> ff = ee.getValue(); > > >> while (ff.size() > 0) { > > >> Entry<byte[], byte[]> cell = ff.pollFirstEntry(); > > >> String key = new String(cell.getKey()); > > >> String val = new String(cell.getValue()); > > >> System.out.println(Thread.currentThread().hashCode() + "\t" > > >> + rowKey + "\t" + fName + "\t" + key + "\t" + val); > > >> } > > >> } > > >> } > > >> } > > >> > > > > > >
