[
https://issues.apache.org/jira/browse/HBASE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13527287#comment-13527287
]
stack commented on HBASE-6466:
------------------------------
bq. The question is, can the patch now go into trunk?
Yes. Default is single flusher only so default behavior should be as it was.
Looking at the patch, please look elsewhere in code base for how we set threads
running. See how the threads are named... there is some convention in that
threads have the name of their host as a prefix which helps when many
regionservers in the one jvm. Could these new flusher be set up using a thread
pool instead? See Threads.java for some facility.
Can this class be static?
+ private class FlushHandler extends HasThread {
Pass in the Service Interface so you can query if host has been stopped?
If we failed a flush in the past, we'd check the filesystem and if we couldn't
write, we'd abort the server. Does that happen still? Or does the flusher
thread just exit?
What is going on w/ blockSignal?
Why in FSHLog change lock to be a ReentrantReadWriteLock?
How are J-Ds concerns above being addressed in this patch? Or Elliott's seeing
60 second pauses? Does this do what Jimmy suggests above, flush multiple
regions concurrently when there is memory pressure?
Thanks
> Enable multi-thread for memstore flush
> --------------------------------------
>
> Key: HBASE-6466
> URL: https://issues.apache.org/jira/browse/HBASE-6466
> Project: HBase
> Issue Type: Improvement
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6466.patch, HBASE-6466v2.patch,
> HBASE-6466v3.1.patch, HBASE-6466v3.patch
>
>
> If the KV is large or Hlog is closed with high-pressure putting, we found
> memstore is often above the high water mark and block the putting.
> So should we enable multi-thread for Memstore Flush?
> Some performance test data for reference,
> 1.test environment ：
> random writting；upper memstore limit 5.6GB;lower memstore limit 4.8GB;400
> regions per regionserver；row len=50 bytes, value len=1024 bytes;5
> regionserver, 300 ipc handler per regionserver;5 client, 50 thread handler
> per client for writing
> 2.test results:
> one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per
> regionserver, appears many aboveGlobalMemstoreLimit blocking
> two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per
> regionserver,
> 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per
> regionserver, Flush:18.6MB/s per regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
