[
https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13285496#comment-13285496
]
chunhui shen commented on HBASE-6134:
-
@ted
In my opition, we could do the improvement for split-worker.
In the patch , I change the split logic which executed by the split-worker same
as local-master-split.
It means, there are multi threads to creater writer and append hlog entry.
Also, we found closing writer is quite slow, so make the closeStreams()
executed by multi threads.
With the patch, we test the time of distribued split log again, and reduce the
time from 165s+ to 38s+
(34hlog files 4 regionervers)
Speed up distribued split log
-
Key: HBASE-6134
URL: https://issues.apache.org/jira/browse/HBASE-6134
Project: HBase
Issue Type: Improvement
Components: wal
Reporter: chunhui shen
Assignee: chunhui shen
Fix For: 0.96.0
Attachments: HBASE-6134.patch
First，we do the test between local-master-split and distributed split log
Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths
splitting work)
local-master-split:60s+
distributed-split-log:165s+
In fact, in our production environment, distributed-split-log also took 60s
with 30 regionservers for 34 hlog files (regionserver may be in high load)
We found split-worker split one log file took about 20s.
I think we should do the improvement for this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
