[
https://issues.apache.org/jira/browse/HBASE-9775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13816831#comment-13816831
]
stack commented on HBASE-9775:
------------------------------
Playing w/ YCSB, I see that as is, we are pretty well-behaved now. A single
client will grow its threads to just under two per server and would hold there
roughly. Sometimes it will expand a little beyond this but these are threads
that are just waiting to be dropped by the pool. On my small cluster of 5
nodes, upping the clients to 8 on a 16 core CPU, I was doing about 600% burn
and the throughput was at just over twice the single thread. To be continued.
I made this change in YCSB to see if more connections would get me more
throughput:
{code}
@@ -117,7 +122,8 @@ public class HBaseClient extends com.yahoo.ycsb.DB
public void getHTable(String table) throws IOException
{
synchronized (tableLock) {
- _hTable = new HTable(config, table);
+ _hConnection = HConnectionManager.createConnection(config);
+ _hTable = _hConnection.getTable(table);
//2 suggestions from
http://ryantwopointoh.blogspot.com/2009/01/performance-of-hbase-importing.html
_hTable.setAutoFlush(false);
_hTable.setWriteBufferSize(1024*1024*12);
{code}
What I saw was that my client now had 2k+ threads running in it. All but a few
were just idle waiting, doing nothing. The burn was up, around 800%. Didn't
bother checking throughput.
So for Elliott test above, he probably had sane number of threads in his test.
But if folks follow our new receipe where they do
createConnection().getTable(TableName) then they will have clients w/ at least
256 threads just sitting there hanging out. Let me fix that one.
> Client write path perf issues
> -----------------------------
>
> Key: HBASE-9775
> URL: https://issues.apache.org/jira/browse/HBASE-9775
> Project: HBase
> Issue Type: Bug
> Components: Client
> Affects Versions: 0.96.0
> Reporter: Elliott Clark
> Priority: Critical
> Attachments: 9775.rig.txt, 9775.rig.v2.patch, 9775.rig.v3.patch,
> Charts Search Cloudera Manager - ITBLL.png, Charts Search Cloudera
> Manager.png, hbase-9775.patch, job_run.log, short_ycsb.png, ycsb.png,
> ycsb_insert_94_vs_96.png
>
>
> Testing on larger clusters has not had the desired throughput increases.
--
This message was sent by Atlassian JIRA
(v6.1#6144)
