[
https://issues.apache.org/jira/browse/HDFS-3077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13467407#comment-13467407
]
Sanjay Radia commented on HDFS-3077:
------------------------------------
bq. Section 2.10.6
{quote}
Call finalizeSegment(1-150) on all JNs, they all succeed
Call startLogSegment(151) on all JNs, they all succeed
Call logEdits(151-153), but it only goes to JN1 before crashing
{quote}
Based on your description, 2.10.6 should be labelled as "Inconsistency on
first batch of log - prior quorum succeeded.
bq. I prefer to think of recovery as having one job: closing off the latest
log segment. At that point, the writer continues on with writing the next
segment using the usual APIs.
I don't have a problem with the newSegment creating the no-op transaction and
then the writer continuing using the normal APIs.
What I dislike is that in example in 2.10.6, recovery completed sucessfully,
the new writer starts writing normally and then fails - you delete a segment
that was successfully finalized at quorum JNs - this is the only example of
successful transaction being deleted. In this situation won't the normal
processing select 151 to be the common transaction and hence result into
segment edits_151_151? Or is it that you can't distinguish between this case
and the case where the quorum failed?
> Quorum-based protocol for reading and writing edit logs
> -------------------------------------------------------
>
> Key: HDFS-3077
> URL: https://issues.apache.org/jira/browse/HDFS-3077
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: ha, name-node
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: QuorumJournalManager (HDFS-3077)
>
> Attachments: hdfs-3077-partial.txt, hdfs-3077-test-merge.txt,
> hdfs-3077.txt, hdfs-3077.txt, hdfs-3077.txt, hdfs-3077.txt, hdfs-3077.txt,
> hdfs-3077.txt, hdfs-3077.txt, qjournal-design.pdf, qjournal-design.pdf,
> qjournal-design.pdf, qjournal-design.pdf, qjournal-design.pdf,
> qjournal-design.tex, qjournal-design.tex
>
>
> Currently, one of the weak points of the HA design is that it relies on
> shared storage such as an NFS filer for the shared edit log. One alternative
> that has been proposed is to depend on BookKeeper, a ZooKeeper subproject
> which provides a highly available replicated edit log on commodity hardware.
> This JIRA is to implement another alternative, based on a quorum commit
> protocol, integrated more tightly in HDFS and with the requirements driven
> only by HDFS's needs rather than more generic use cases. More details to
> follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
