[
https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16297516#comment-16297516
]
Chris Douglas commented on HDFS-12943:
--------------------------------------
Thanks for the document and benchmarking. This is really cool.
Right now, writes are effectively throttled by blocking reads e.g., conditional
checks before doing a rename. So if the NN is under heavy load, most
applications will appear to back off because all these operations are blocking.
If StandbyNodes serve many of these reads, then the write rate to the primary
NameNode will increase. Have you tried running workloads against the PoC to get
a sense for the "natural" increase in write traffic? In some deployments, would
it make sense to disallow reads from the primary to prevent clients from
harming overall cluster throughput?
> Consistent Reads from Standby Node
> ----------------------------------
>
> Key: HDFS-12943
> URL: https://issues.apache.org/jira/browse/HDFS-12943
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs
> Reporter: Konstantin Shvachko
> Attachments: ConsistentReadsFromStandbyNode.pdf
>
>
> StandbyNode in HDFS is a replica of the active NameNode. The states of the
> NameNodes are coordinated via the journal. It is natural to consider
> StandbyNode as a read-only replica. As with any replicated distributed system
> the problem of stale reads should be resolved. Our main goal is to provide
> reads from standby in a consistent way in order to enable a wide range of
> existing applications running on top of HDFS.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
