[
https://issues.apache.org/jira/browse/HDFS-5037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wang updated HDFS-5037:
------------------------------
Resolution: Fixed
Fix Version/s: 2.2.1
Target Version/s: 2.1.0-beta, 3.0.0 (was: 3.0.0, 2.1.0-beta)
Status: Resolved (was: Patch Available)
Committed to trunk, branch-2, branch-2.2. Had to do some little fixups for
branch-2/2.2, wasn't an entirely clean application.
> Active NN should trigger its own edit log rolls
> -----------------------------------------------
>
> Key: HDFS-5037
> URL: https://issues.apache.org/jira/browse/HDFS-5037
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: ha, namenode
> Affects Versions: 3.0.0, 2.1.0-beta
> Reporter: Todd Lipcon
> Assignee: Andrew Wang
> Priority: Critical
> Fix For: 2.2.1
>
> Attachments: hdfs-5037-1.patch, hdfs-5037-2.patch, hdfs-5037-3.patch,
> hdfs-5037-4.patch, hdfs-5037-5.patch
>
>
> We've seen cases where the SBN/2NN went down, and then users accumulated very
> very large edit log segments. This causes a slow startup time because the
> last edit log segment must be read fully to recover it before the NN can
> start up again. Additionally, in the case of QJM, it can trigger timeouts on
> recovery or edit log syncing because the very-large segment has to get
> processed within a certain time bound.
> We could easily improve this by having the NN trigger its own edit log rolls
> on a configurable size (eg every 256MB)
--
This message was sent by Atlassian JIRA
(v6.1#6144)
