[
https://issues.apache.org/jira/browse/HDFS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-5037:
------------------------------
Attachment: hdfs-5037-5.patch
New patch attached addressing your comments, thanks ATM. I think the test case
was safe before since it the standby defaults to checkpointing every hour and
starts by sleeping, but I also tweaked the values for further safety.
Will commit shortly.
> Active NN should trigger its own edit log rolls
> -----------------------------------------------
>
> Key: HDFS-5037
> URL: https://issues.apache.org/jira/browse/HDFS-5037
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha, namenode
> Affects Versions: 3.0.0, 2.1.0-beta
> Reporter: Todd Lipcon
> Assignee: Andrew Wang
> Priority: Critical
> Attachments: hdfs-5037-1.patch, hdfs-5037-2.patch, hdfs-5037-3.patch,
> hdfs-5037-4.patch, hdfs-5037-5.patch
>
>
> We've seen cases where the SBN/2NN went down, and then users accumulated very
> very large edit log segments. This causes a slow startup time because the
> last edit log segment must be read fully to recover it before the NN can
> start up again. Additionally, in the case of QJM, it can trigger timeouts on
> recovery or edit log syncing because the very-large segment has to get
> processed within a certain time bound.
> We could easily improve this by having the NN trigger its own edit log rolls
> on a configurable size (eg every 256MB)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
