[
https://issues.apache.org/jira/browse/HBASE-6466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
chunhui shen updated HBASE-6466:
--------------------------------
Description:
If the KV is large or Hlog is closed with high-pressure putting, we found
memstore is often above the high water mark and block the putting.
So should we enable multi-thread for Memstore Flush?
Some performance test data for reference,
1.test environment ：
random writting；400 regions per regionserver；row len=50 bytes, value len=1024
bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thread
handler per client for writing
2.test results:
one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per
regionserver, appears many aboveGlobalMemstoreLimit blocking
two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per
regionserver,
200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per
regionserver, Flush:18.6MB/s per regionserver
was:
If the KV is large or Hlog is closed with high-pressure putting, we found
memstore is often above the high water mark and block the putting.
So should we enable multi-thread for Memstore Flush?
Some performance test data for reference,
1.test environment ：
random writting；400 regions per regionserver；row len=50 bytes, value len=1024
bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thread
handler per client for writing
2.test results:
one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per
regionserver, appears many above global memstore blocking
two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per
regionserver,
200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per
regionserver, Flush:18.6MB/s per regionserver
> Enable multi-thread for memstore flush
> --------------------------------------
>
> Key: HBASE-6466
> URL: https://issues.apache.org/jira/browse/HBASE-6466
> Project: HBase
> Issue Type: Improvement
> Reporter: chunhui shen
> Assignee: chunhui shen
> Attachments: HBASE-6466.patch, HBASE-6466v2.patch, HBASE-6466v3.patch
>
>
> If the KV is large or Hlog is closed with high-pressure putting, we found
> memstore is often above the high water mark and block the putting.
> So should we enable multi-thread for Memstore Flush?
> Some performance test data for reference,
> 1.test environment ：
> random writting；400 regions per regionserver；row len=50 bytes, value len=1024
> bytes;5 regionserver, 300 ipc handler per regionserver;5 client, 50 thread
> handler per client for writing
> 2.test results:
> one cacheFlush handler, tps: 7.8k/s per regionserver, Flush:10.1MB/s per
> regionserver, appears many aboveGlobalMemstoreLimit blocking
> two cacheFlush handlers, tps: 10.7k/s per regionserver, Flush:12.46MB/s per
> regionserver,
> 200 thread handler per client & two cacheFlush handlers, tps:16.1k/s per
> regionserver, Flush:18.6MB/s per regionserver
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
