[
https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721595#comment-16721595
]
Erik Krogen edited comment on HDFS-12943 at 12/14/18 4:50 PM:
--------------------------------------------------------------
Hey [~brahmareddy], thanks for trying it out and for the detailed feedback!
I think when we discuss a "request", we need to differentiate an RPC request
originating from a Java application (MapReduce task, etc.) vs. a CLI request.
The former will be the vast majority of operations on a typical cluster, so I
would argue that optimizing for the performance and efficiency of that usage is
much more important. The ObserverReadProxyProvider does have higher startup
overheads as it directly polls for the state rather than just blindly trying
its request; however, in an application which performs more than a few RPCs,
this cost will be easily amortized away. I don't think it's fair to say that
"write" performance is degraded simply because {{hdfs dfs -mkdirs}} takes
longer; a benchmark running 100+ mkdirs would be a better measure IMO. If CLI
performance is important, such clients can continue to use
ConfiguredFailoverProxyProvider and communicate with the active directly.
The timeout you have shared is interesting. I suspect that it may be caused by
the Observer trying to wait for its state to catch up to the stateID requested
by your getFileInfo. I have a few questions:
# Are you running with HDFS-13873? With this patch (only committed yesterday so
I doubt you have it) the exception thrown should be more meaningful.
# Did you remember to enable in-progress edit log tailing?
# Was this run on an almost completely stagnant cluster (no other writes)? This
can make the ANN flush its edits to the JNs less frequently, increasing the lag
time between ANN and Observer.
was (Author: xkrogen):
Hey [~brahmareddy], thanks for trying it out and for the detailed feedback!
I think when we discuss a "request", we need to differentiate an RPC request
originating from a Java application (MapReduce task, etc.) vs. a CLI request.
The former will be the vast majority of operations on a typical cluster, so I
would argue that optimizing for the performance and efficiency of that usage is
much more important. The ObserverReadProxyProvider does have higher startup
overheads as it directly polls for the state rather than just blindly trying
its request; however, in an application which performs more than a few RPCs,
this cost will be easily amortized away. I don't think it's fair to say that
"write" performance is degraded simply because {{hdfs dfs -mkdirs}} takes
longer; a benchmark running 100+ mkdirs would be a better measure IMO.
The timeout you have shared is interesting. I suspect that it may be caused by
the Observer trying to wait for its state to catch up to the stateID requested
by your getFileInfo. I have a few questions:
# Are you running with HDFS-13873? With this patch (only committed yesterday so
I doubt you have it) the exception thrown should be more meaningful.
# Did you remember to enable in-progress edit log tailing?
# Was this run on an almost completely stagnant cluster (no other writes)? This
can make the ANN flush its edits to the JNs less frequently, increasing the lag
time between ANN and Observer.
> Consistent Reads from Standby Node
> ----------------------------------
>
> Key: HDFS-12943
> URL: https://issues.apache.org/jira/browse/HDFS-12943
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs
> Reporter: Konstantin Shvachko
> Priority: Major
> Attachments: ConsistentReadsFromStandbyNode.pdf,
> ConsistentReadsFromStandbyNode.pdf, HDFS-12943-001.patch,
> TestPlan-ConsistentReadsFromStandbyNode.pdf
>
>
> StandbyNode in HDFS is a replica of the active NameNode. The states of the
> NameNodes are coordinated via the journal. It is natural to consider
> StandbyNode as a read-only replica. As with any replicated distributed system
> the problem of stale reads should be resolved. Our main goal is to provide
> reads from standby in a consistent way in order to enable a wide range of
> existing applications running on top of HDFS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
