[
https://issues.apache.org/jira/browse/HBASE-6134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285496#comment-13285496
]
chunhui shen commented on HBASE-6134:
-------------------------------------
@ted
In my opition, we could do the improvement for split-worker.
In the patch , I change the split logic which executed by the split-worker same
as local-master-split.
It means, there are multi threads to creater writer and append hlog entry.
Also, we found closing writer is quite slow, so make the closeStreams()
executed by multi threads.
With the patch, we test the time of distribued split log again, and reduce the
time from 165s+ to 38s+
(34hlog files 4 regionervers)
> Speed up distribued split log
> -----------------------------
>
> Key: HBASE-6134
> URL: https://issues.apache.org/jira/browse/HBASE-6134
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Reporter: chunhui shen
> Assignee: chunhui shen
> Fix For: 0.96.0
>
> Attachments: HBASE-6134.patch
>
>
> First，we do the test between local-master-split and distributed split log
> Environment：34 hlog files, 5 regionservers,(after kill one, only 4 rs do ths
> splitting work)
> local-master-split:60s+
> distributed-split-log:165s+
> In fact, in our production environment, distributed-split-log also took 60s
> with 30 regionservers for 34 hlog files (regionserver may be in high load)
> We found split-worker split one log file took about 20s.
> I think we should do the improvement for this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
