I was just thinking on this. Testing, changing the index interval has
the biggest direct effect on memory used. Currently its 32 which seems
to be fine for cells of 1K and greater. With 32, you can load a decent
amount of regions per regionserver. Smaller cells seem common enough
though. Here interval should be the hadoop, as opposed to hbase,
default of 128 or even 1024 (Its 'low' to help improve access times).
It seems that for the case where cells are < 10 bytes or so, interval
should be 1k or even 4k. Should we just be conservative in defaults
since OOME is worse than slow access times.
Backporting HBASE-900 might be a bit much for 0.18.2. We should for
sure backport the fix that makes it so you can set the index interval to
0.18 branch.
St.Ack
Andrew Purtell wrote:
Does it make sense to backport HBASE-900 to 0.18.2?
- Andy
From: Ryan LeCompte <[email protected]>
Subject: Re: Region server memory requirements
To: [email protected]
Date: Friday, December 19, 2008, 1:06 PM
So, I tried giving each of the region servers 2GB and also
tried to limit the number of cells/columns I'm creating,
but the memory still maxes out at 2GB eventually runs into
OOME.
