[
https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16725379#comment-16725379
]
Chen Liang commented on HDFS-12943:
-----------------------------------
Hi [~brahmareddy],
Some more notes to add:
1. getHAServiceState() only gets called when initialization of client proxies
(and of course when existing proxies failed and client reinitialize them). In
regular operation, this call will not happen so it should not be a concern in
benchmarks.
2. I tried the unit test you shared locally with Observer read
enabled/disabled. I did not see difference in terms of mkdir time, it has been
about 2ms the whole time regardless. I saw some degradation on get content
summary though. But this is due to that the unit test is doing mkdir ->
getContentSummary -> getFileStatus -> repeat. So the client is constantly
switching between write and read, and thus constantly switching between
proxies(NNs). This is not the IO pattern Observer is mainly targeting for, and
probably the worst case for Observer read because every single
getContentSummary call here could potentially trigger Observer catch up wait.
> Consistent Reads from Standby Node
> ----------------------------------
>
> Key: HDFS-12943
> URL: https://issues.apache.org/jira/browse/HDFS-12943
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs
> Reporter: Konstantin Shvachko
> Priority: Major
> Attachments: ConsistentReadsFromStandbyNode.pdf,
> ConsistentReadsFromStandbyNode.pdf, HDFS-12943-001.patch,
> HDFS-12943-002.patch, TestPlan-ConsistentReadsFromStandbyNode.pdf
>
>
> StandbyNode in HDFS is a replica of the active NameNode. The states of the
> NameNodes are coordinated via the journal. It is natural to consider
> StandbyNode as a read-only replica. As with any replicated distributed system
> the problem of stale reads should be resolved. Our main goal is to provide
> reads from standby in a consistent way in order to enable a wide range of
> existing applications running on top of HDFS.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
