[
https://issues.apache.org/jira/browse/HDFS-13189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16830014#comment-16830014
]
xuzq commented on HDFS-13189:
-----------------------------
I think the edit log roll and tailing edits need to do in separate threads in
Standby, because they all independency.
There is one other problem to solve. In current trunk, both Observers and
Standbys can rollEdit, it may produce too many small segments. And will
produce some negative effects, such as too many small files in each
JournalNode.
> Standby NameNode should roll active edit log when checkpointing
> ---------------------------------------------------------------
>
> Key: HDFS-13189
> URL: https://issues.apache.org/jira/browse/HDFS-13189
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Chao Sun
> Priority: Minor
>
> When the SBN is doing checkpointing, it will hold the {{cpLock}}. In the
> current implementation of edit log tailer thread, it will first check and
> roll active edit log, and then tail and apply edits. In the case of
> checkpointing, it will be blocked on the {{cpLock}} and will not roll the
> edit log.
> It seems there is no dependency between the edit log roll and tailing edits,
> so a better may be to do these in separate threads. This will be helpful for
> people who uses the observer feature without in-progress edit log tailing.
> An alternative is to configure
> {{dfs.namenode.edit.log.autoroll.multiplier.threshold}} and
> {{dfs.namenode.edit.log.autoroll.check.interval.ms}} to let ANN roll its own
> log more frequently in case SBN is stuck on the lock.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
