terrible ...
org.apache.hadoop.hbase.client.ScannerTimeoutException: 338424ms passed
since the last invocation, timeout is currently set to 30
On Tue, Feb 1, 2011 at 6:45 AM, Wayne wav...@gmail.com wrote:
The file system buffer cache explains what is going on. The open scanner
reads the
Hey,
The region location cache is held by a soft reference, so as long as
you dont have memory pressure, it will never get invalidated just
because of time.
Another thing to consider, in HBase, the open scanner code also seeks
and reads the first block of the scan. This may incur a read to disk
We have heavy writes always going on so there is always memory pressure.
If the open scanner reads the first block maybe that explains the 8ms the
second time a test is run, but why is the first run averaging 35ms to open
and when the same read requests are sent again the open is only 8ms? There
On Mon, Jan 31, 2011 at 1:38 PM, Wayne wav...@gmail.com wrote:
After doing many tests (10k serialized scans) we see that on average opening
the scanner takes 2/3 of the read time if the read is fresh
(scannerOpenWithStop=~35ms, scannerGetList=~10ms).
I've saw that this w/e. The getScanner
The Regionserver caches blocks, so a second read would benefit from
the caching of the first read. Over time blocks get evicted in a LRU
manner, and things would get slow again.
Does this make sense to you?
On Mon, Jan 31, 2011 at 1:50 PM, Wayne wav...@gmail.com wrote:
We have heavy writes
On Mon, Jan 31, 2011 at 4:54 PM, Stack st...@duboce.net wrote:
On Mon, Jan 31, 2011 at 1:38 PM, Wayne wav...@gmail.com wrote:
After doing many tests (10k serialized scans) we see that on average
opening
the scanner takes 2/3 of the read time if the read is fresh
I assume BLOCKCACHE = 'false' would turn this off? We have turned off cache
on all tables.
On Mon, Jan 31, 2011 at 4:54 PM, Ryan Rawson ryano...@gmail.com wrote:
The Regionserver caches blocks, so a second read would benefit from
the caching of the first read. Over time blocks get evicted in
Even without block caching, the linux buffer cache is still a factor,
and your reads still go through them (via the datanode).
When Stack is talking about the StoreScanner, this is a particular
class inside of HBase that does the job of reading from 1 column
family. The first time you
The file system buffer cache explains what is going on. The open scanner
reads the first block and the subsequent read goes against the same block
thereby getting out of the file buffer cache.
Thanks.
On Mon, Jan 31, 2011 at 5:22 PM, Ryan Rawson ryano...@gmail.com wrote:
Even without block
