stack created HBASE-15594:
-----------------------------
Summary: [YCSB] Improvements
Key: HBASE-15594
URL: https://issues.apache.org/jira/browse/HBASE-15594
Project: HBase
Issue Type: Umbrella
Reporter: stack
Running YCSB and getting good results is an arcane art. For example, in my
testing, a few handlers (100) with as many readers as I had CPUs (48), and
upping connections on clients to same as #cpus made for 2-3x the throughput.
The above config changes came of lore; which configurations need tweaking is
not obvious going by their names, there were no indications from the app on
where/why we were blocked or on which metrics are important to consider. Nor
was any of this stuff written down in docs.
Even still, I am stuck trying to make use of all of the machine. I am unable to
overrun a server though 8 client nodes trying to beat up a single node
(workloadc, all random-read, with no data returned -p readallfields=false).
There is also a strange phenomenon where if I add a few machines, rather than
3x the YCSB throughput when 3 nodes in cluster, each machine instead is doing
about 1/3rd.
This umbrella issue is to host items that improve our defaults and noting how
to get good numbers running YCSB. In particular, I want to be able to saturate
a machine.
Here are the configs I'm currently working with. I've not done the work to
figure client-side if they are optimal (weird is how big a difference
client-side changes can make -- need to fix this). On my 48 cpu machine, I can
do about 370k random reads a second from data totally cached in bucketcache. If
I short-circuit the user gets so they don't do any work but return immediately,
I can do 600k ops a second but the CPUs are at 60-70% only. I cannot get them
to go above this. Working on it.
{code}
<property>
<name>
hbase.ipc.server.read.threadpool.size
</name>
<value>48</value>
</property>
<property>
<name>
hbase.regionserver.handler.count
</name>
<value>100</value>
</property>
<property>
<name>
hbase.client.ipc.pool.size
</name>
<value>100</value>
</property>
<property>
<name>
hbase.htable.threads.max
</name>
<value>48</value>
</property>
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
