[
https://issues.apache.org/jira/browse/HDFS-8178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhe Zhang updated HDFS-8178:
----------------------------
Description: When a QJM crashes, the in-progress edit log file at that time
remains in the file system. When the node comes back, it will accept new edit
logs and those stale in-progress files are never cleaned up. QJM treats them as
regular in-progress edit log files and tries to finalize them, which
potentially causes high memory usage. This JIRA aims to move aside those stale
edit log files to avoid this scenario. (was: HDFS-5919 fixes the issue for
{{FileJournalManager}}. A similar fix is needed for QJM.)
> QJM doesn't purge empty and corrupt inprogress edits files
> ----------------------------------------------------------
>
> Key: HDFS-8178
> URL: https://issues.apache.org/jira/browse/HDFS-8178
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: qjm
> Reporter: Zhe Zhang
> Assignee: Zhe Zhang
> Attachments: HDFS-8178.000.patch
>
>
> When a QJM crashes, the in-progress edit log file at that time remains in the
> file system. When the node comes back, it will accept new edit logs and those
> stale in-progress files are never cleaned up. QJM treats them as regular
> in-progress edit log files and tries to finalize them, which potentially
> causes high memory usage. This JIRA aims to move aside those stale edit log
> files to avoid this scenario.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
