[
https://issues.apache.org/jira/browse/HDFS-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17408479#comment-17408479
]
liutongwei commented on HDFS-13150:
-----------------------------------
[~xkrogen] As I'm learning the design doc of fast path tailing, I have a doubt
about the correctness of the approach 1 to ensure only committed transactions
are applied.
In this design, the minimum lastWrittenTxId is used to get a safe point to
applied log. But in some corner case, if there is a out-synced JN back online,
it may indeed got the minimum lastWrittenTxId, but the lastWrittenTxId's data
in this JN may differ from other JNs. This may because the lastWrittenTxId's in
the out-synced JN is not committed by the prior writer, and was overwritten by
new epoch writer.In this case, we got uncommitted data.
How about use a quorum read combined approach 2 to get max committed txid, it
definitely correct because committed txid is updated by the writer, it was
guaranteed by the writer even a recovery was occurred.
Correct me if anything is wrong.
Looking forward for you reply.
> [Edit Tail Fast Path] Allow SbNN to tail in-progress edits from JN via RPC
> --------------------------------------------------------------------------
>
> Key: HDFS-13150
> URL: https://issues.apache.org/jira/browse/HDFS-13150
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: ha, hdfs, journal-node, namenode
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Priority: Major
> Fix For: HDFS-12943, 3.3.0
>
> Attachments: edit-tailing-fast-path-design-v0.pdf,
> edit-tailing-fast-path-design-v1.pdf, edit-tailing-fast-path-design-v2.pdf
>
>
> In the interest of making coordinated/consistent reads easier to complete
> with low latency, it is advantageous to reduce the time between when a
> transaction is applied on the ANN and when it is applied on the SbNN. We
> propose adding a new "fast path" which can be used to tail edits when low
> latency is desired. We leave the existing tailing logic in place, and fall
> back to this path on startup, recovery, and when the fast path encounters
> unrecoverable errors.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
