[
https://issues.apache.org/jira/browse/HDFS-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397731#comment-16397731
]
Konstantin Shvachko commented on HDFS-13150:
--------------------------------------------
As a reminder, the two approaches for SBN / ObserverNode reading from Journal
nodes are:
# SBN *reads from a quorum* of JNs
# SBN *reads from single* JN, while JNs guarantee serving only committed
transactions
I am advocating that approach 2 is faster. Suppose we have 3 journal nodes.
* When SBN reads from the quorum (approach 1) it updates its state as fast as
the second slowest JN.
* With approach 2 we can choose the fastest JN most of the time. By
periodically polling JNs and switching to the one that has higher txId.
There is an issue of confirming committed transactions from ANN to JNs. But
every next batch of edits sent by ANN to a JN essentially confirms that the
previous batch is committed. This does not require extra dummy syncs. Under
regular load ANN will be sending batches of edits continuously, so JNs will be
up-to-date up to the last processed batch. ANN will occasionally need to send
the extra “dummy” sync, but it is required only if ANN doesn't have load at all
or no writes.
_Having said that, I am fine with quorum reads as the initial implementation
if it is simpler, as I was told._
> Create fast path for SbNN tailing edits from JNs
> ------------------------------------------------
>
> Key: HDFS-13150
> URL: https://issues.apache.org/jira/browse/HDFS-13150
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs, journal-node, namenode
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Priority: Major
> Attachments: edit-tailing-fast-path-design-v0.pdf,
> edit-tailing-fast-path-design-v1.pdf
>
>
> In the interest of making coordinated/consistent reads easier to complete
> with low latency, it is advantageous to reduce the time between when a
> transaction is applied on the ANN and when it is applied on the SbNN. We
> propose adding a new "fast path" which can be used to tail edits when low
> latency is desired. We leave the existing tailing logic in place, and fall
> back to this path on startup, recovery, and when the fast path encounters
> unrecoverable errors.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
