[
https://issues.apache.org/jira/browse/HDFS-3900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated HDFS-3900:
------------------------------
Attachment: hdfs-3900.txt
Attached patch fixes the issue. I also did some code cleanup around the
treatment of the {{curSegmentTxId}} variable, which was used somewhat
inconsistently before. Now it's only set when a segment is open, which is a
little cleaner.
> QJM: avoid validating log segments on log rolls
> -----------------------------------------------
>
> Key: HDFS-3900
> URL: https://issues.apache.org/jira/browse/HDFS-3900
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: QuorumJournalManager (HDFS-3077)
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: hdfs-3900.txt
>
>
> Currently, we are paranoid and validate every log segment when it is
> finalized. For the a log segment that has been written entirely by one
> writer, with no recovery in between, this is overly paranoid (we don't do
> this for local journals). It also causes log rolls to be slow and take time
> linear in the size of the segment. Instead, we should optimize this path to
> simply trust that the segment is correct so long as the txids match up as
> expected.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
