[
https://issues.apache.org/jira/browse/HDFS-13150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16377444#comment-16377444
]
Chao Sun commented on HDFS-13150:
---------------------------------
Thanks for the design doc [~xkrogen]! The doc looks great overall. I have a few
comments after reading it:
* what is the relation of this with the in-progress edit log tailing? also, is
this a separate feature that can be turned on/off? or we target this as a
replacement for the current approach?
* how much savings did you see by performing preloading edits outside the
lock? I did something similar in my experiment but it didn't show obvious
benefit.
* with the cache, does it mean we need to extra decode + encode on the journal
side? is there any perf impact on the journal?
* in the performance evaluation section, the max(ms) in scenario 5 is
extremely high comparing to others, is that a mistake?
* To achieve low latency the Observer NameNode also needs to pull from the
journal nodes in a very high frequency right? did you cover that in the
benchmark?
* might be better to explain a little that on the high-level how this can be
used for Observer NameNode.
> Create fast path for SbNN tailing edits from JNs
> ------------------------------------------------
>
> Key: HDFS-13150
> URL: https://issues.apache.org/jira/browse/HDFS-13150
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: hdfs, journal-node, namenode
> Reporter: Erik Krogen
> Assignee: Erik Krogen
> Priority: Major
> Attachments: edit-tailing-fast-path-design-v0.pdf
>
>
> In the interest of making coordinated/consistent reads easier to complete
> with low latency, it is advantageous to reduce the time between when a
> transaction is applied on the ANN and when it is applied on the SbNN. We
> propose adding a new "fast path" which can be used to tail edits when low
> latency is desired. We leave the existing tailing logic in place, and fall
> back to this path on startup, recovery, and when the fast path encounters
> unrecoverable errors.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
