Zhe Zhang created HDFS-8964:
-------------------------------
Summary: Provide max TxId when validating in-progress edit log
files
Key: HDFS-8964
URL: https://issues.apache.org/jira/browse/HDFS-8964
Project: Hadoop HDFS
Issue Type: Bug
Components: journal-node, namenode
Affects Versions: 2.7.1
Reporter: Zhe Zhang
Assignee: Zhe Zhang
NN/JN validates in-progress edit log files in multiple scenarios, via
{{EditLogFile#validateLog}}. The method scans through the edit log file to find
the last transaction ID.
However, an in-progress edit log file could be actively written to, which
creates a race condition and causes incorrect data to be read (and later we
attempt to interpret the data as ops).
Currently {{validateLog}} is used in 3 places:
# NN {{getEditsFromTxid}}
# JN {{getEditLogManifest}}
# NN/JN {{recoverUnfinalizedSegments}}
In the first two scenarios we should provide a maximum TxId to validate in the
in-progress file. The 3rd scenario won't cause a race condition because only
non-current in-progress edit log files are validated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
