[
https://issues.apache.org/jira/browse/HBASE-19358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16279909#comment-16279909
]
Yu Li commented on HBASE-19358:
-------------------------------
Thanks for the tests [~tianjingyun], but after a closer look, I found I've
asked the wrong question since
{{hbase.regionserver.hlog.splitlog.writer.threads}} is an existing property
rather than newly introduced one.
So basically the idea proposed in this JIRA is like below (please correct me if
I stated anything wrong):
# Currently we'll start writing data into recovered.edits directory as soon as
split starts, while after patch we will set some "entry flush size" and only
start writing entries after {{WALSplitter#buffers}} reach this size
# Currently we won't close any wap (WriteAndPath) until split done, which will
cause plenty of open pipelines to DN, while after patch we will close the wap
at end of each flush round (or in another word, we possibly will generate more
files under recovered.edits for regions with heavy writes)
If the above descriptions are right, I think the new design won't cause big
regression in recovery time for cluster with light load (few regions on each
RS), or at least we could tune it by setting a correct "entry flush size".
And if the above descriptions are right, I'd suggest to update the design doc
to better reflect the idea. The current one would possibly make people think
the core design is to introduce a thread pool while actually it's not.
Thanks.
> Improve the stability of splitting log when do fail over
> --------------------------------------------------------
>
> Key: HBASE-19358
> URL: https://issues.apache.org/jira/browse/HBASE-19358
> Project: HBase
> Issue Type: Improvement
> Components: MTTR
> Affects Versions: 0.98.24
> Reporter: Jingyun Tian
> Assignee: Jingyun Tian
> Attachments: newLogic.jpg, previousLogic.jpg, split-1-log.png,
> split-table.png, split_test_result.png
>
>
> The way we splitting log now is like the following figure:
> !https://issues.apache.org/jira/secure/attachment/12899558/previousLogic.jpg!
> The problem is the OutputSink will write the recovered edits during splitting
> log, which means it will create one WriterAndPath for each region. If the
> cluster is small and the number of regions per rs is large, it will create
> too many HDFS streams at the same time. Then it is prone to failure since
> each datanode need to handle too many streams.
> Thus I come up with a new way to split log.
> !https://issues.apache.org/jira/secure/attachment/12899557/newLogic.jpg!
> We cached the recovered edits unless exceeds the memory limits we set or
> reach the end, then we have a thread pool to do the rest things: write them
> to files and move to the destination.
> The biggest benefit is we can control the number of streams we create during
> splitting log,
> it will not exceeds *_hbase.regionserver.wal.max.splitters *
> hbase.regionserver.hlog.splitlog.writer.threads_*, but before it is
> *_hbase.regionserver.wal.max.splitters * the number of region the hlog
> contains_*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
