[
https://issues.apache.org/jira/browse/HBASE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13528726#comment-13528726
]
stack commented on HBASE-6466:
------------------------------
bq. I guess one could achieve the same goal with threadpool and counter to
limit the concurrency (except that these threads will be easier to starve) but
it seems like a roundabout way to do is.
Thanks for taking a look. The diff between thread pool and what is running
here makes sense.
bq. At the cost of few pointers (it currently refers to parent fields
Thats ok. Was just asking. Its fine the way it is especially later where you
point out that you are making use of Server from the parent class.
bq. The behavior should be the same - the uncaught exception handler is set on
new runnable-s the same way as on the old one.
Ok. Was worried the exception would be dropped on the floor. Sounds good.
bq. ....WAL has entries for cache flush (which according to discussion in the
linked FB JIRA might be unnecessary)...
Yeah, we don't make use of these flush markers.
bq. What is going on w/ blockSignal?
I ask because I'm wary when new locks added. This is an addition in a little
exercised piece of the code. Was looking for justification on why the
addition. On the face of it, it seems plain enough.
bq. ... with lock held between start and complete entries. If this lock is kept
exclusive, it will cause flush threads to serialize on it.
Ok. Were you able to make this happen Sergey?
On jds' concern, its this one: 'Also if this patch doesn't modify the behavior
of HLog.startCacheFlush and HLog.completeCacheFlush WRT the cacheFlushLock I
can't see how it could make things any faster.'
So, your reentrant lock is how you address his concern?
Any guards against us flushing same memstore concurrently: i.e. we are already
flushing it and we start in flushing it again in a concurrent thread?
> Enable multi-thread for memstore flush
> --------------------------------------
>
> Key: HBASE-6466
> URL: https://issues.apache.org/jira/browse/HBASE-6466
> Project: HBase
> Issue Type: Improvement
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6466.patch, HBASE-6466v2.patch,
> HBASE-6466v3.1.patch, HBASE-6466v3.patch, HBASE-6466-v4.patch
>
>
> If the KV is large or Hlog is closed with high-pressure putting, we found
> memstore is often above the high water mark and block the putting.
> So should we enable multi-thread for Memstore Flush?
> Some performance test data for reference,
> 1.test environment ：
> random writting；upper memstore limit 5.6GB;lower memstore limit 4.8GB;400
> regions per regionserver；row len=50 bytes, value len=1024 bytes;5
> regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler
> per client for writing
> 2.test results:
> one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per
> regionserver, appears many aboveGlobalMemstoreLimit blocking
> two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per
> regionserver,
> 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per
> regionserver, Flush:18.6MB/s per regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
