Hey,
We are using intel i7 machines with dual quad htt, 24 gb ram. That
makes for 16 processors and 4 * 1tb disks. I also use the following
settings in hbase-site.xml:
<property>
<name>hbase.hregion.memstore.block.multiplier</name>
<value>4</value>
<description>
Block updates if memcache has hbase.hregion.block.memcache
time hbase.hregion.flush.size bytes. Useful preventing
runaway memcache during spikes in update traffic. Without an
upper-bound, memcache fills such that when it flushes the
resultant flush files take a long time to compact or split, or
worse, we OOME.
</description>
</property>
<property>
<name>hbase.hstore.blockingStoreFiles</name>
<value>15</value>
</property>
Also note you will want to be running with CMS, a java option like so works:
"-XX:+DoEscapeAnalysis -XX:+AggressiveOpts -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=88 -verbose:gc -XX:+PrintGCDetails
-XX:+PrintGCTimeStamps -Xloggc:/tmp/gc-hbase.log"
to get both verbose logs, aggressive optimizations and CMS with a
lower initiating OCC fraction. I find these settings to work well for
me.
The hard part is getting enough clients to pump enough requests to a
single regionserver. Once your table grows and splits across multiple
servers, this can be a little tricky. There are planned changes that
will help this improve performance.
Good luck!
-ryan
On Sun, Oct 18, 2009 at 7:58 AM, Sylvain Hellegouarch <[email protected]> wrote:
> Ryan Rawson a écrit :
>>
>> On top notch modern hardware, with tweaking, and some client luck, a
>> regionserver can do up to 55k inserts/sec. Maybe more with some
>> tweaks I'm planning.
>>
>
> Ryan, I feel those are a tad vague terms but they raise my curiosity
> considering we, too, reach about 10k inserts/s on a very small cluster and
> we do consider 10k/s to be a very comfortable range already but a 45k
> difference is a massive increase.
>
> What is considered modern hardware in your book? What sorts of tweaking are
> you suggesting? How to tests those tweaks?
>
> You do right mentioning luck because beyond the obvious it seems that all
> the literature on HBase can offer us (and yes I'm aware of the
> PerformanceTunning page ;)).
>
> - Sylvain
>
>
>
