[
https://issues.apache.org/jira/browse/HBASE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13529218#comment-13529218
]
Sergey Shelukhin commented on HBASE-6466:
-----------------------------------------
bq. Yeah, we don't make use of these flush markers.
Created HBASE-7329 to remove.
{quote}
bq. ... with lock held between start and complete entries. If this lock is
kept exclusive, it will cause flush threads to serialize on it.
{quote}
bq. Ok. Were you able to make this happen Sergey?
I took the patch with locks already fixed. I can try but it seems there's
already consensus on this :)
bq. On jds' concern, its this one: 'Also if this patch doesn't modify the
behavior of HLog.startCacheFlush and HLog.completeCacheFlush WRT the
cacheFlushLock I can't see how it could make things any faster.'
bq. So, your reentrant lock is how you address his concern?
Yeah, I think this locking was meant by the comment.
bq. Any guards against us flushing same memstore concurrently: i.e. we are
already flushing it and we start in flushing it again in a concurrent thread?
There's code in flushcache that prevents this via writestate:
{code}
synchronized (writestate) {
if (!writestate.flushing && writestate.writesEnabled) {
this.writestate.flushing = true;
} else {
{code}
close calls internal flush directly but it also sets writesEnabled to false.
> Enable multi-thread for memstore flush
> --------------------------------------
>
> Key: HBASE-6466
> URL: https://issues.apache.org/jira/browse/HBASE-6466
> Project: HBase
> Issue Type: Improvement
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6466.patch, HBASE-6466v2.patch,
> HBASE-6466v3.1.patch, HBASE-6466v3.patch, HBASE-6466-v4.patch
>
>
> If the KV is large or Hlog is closed with high-pressure putting, we found
> memstore is often above the high water mark and block the putting.
> So should we enable multi-thread for Memstore Flush?
> Some performance test data for reference,
> 1.test environment ：
> random writting；upper memstore limit 5.6GB;lower memstore limit 4.8GB;400
> regions per regionserver；row len=50 bytes, value len=1024 bytes;5
> regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler
> per client for writing
> 2.test results:
> one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per
> regionserver, appears many aboveGlobalMemstoreLimit blocking
> two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per
> regionserver,
> 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per
> regionserver, Flush:18.6MB/s per regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
