[
https://issues.apache.org/jira/browse/HDFS-12943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Shvachko updated HDFS-12943:
---------------------------------------
Release Note:
Observer is a new type of a NameNode in addition to Active and Standby Nodes in
HA settings. An Observer Node maintains a replica of the namespace same as a
Standby Node. It additionally allows execution of clients read requests.
To ensure read-after-write consistency within a single client, a state ID is
introduced in RPC headers. The Observer responds to the client request only
after its own state has caught up with the client’s state ID, which it
previously received from the Active NameNode.
Clients can explicitly invoke a new client protocol call msync(), which ensures
that subsequent reads by this client from an Observer are consistent.
A new client-side ObserverReadProxyProvider is introduced to provide automatic
switching between Active and Observer NameNodes for submitting respectively
write and read requests.
was:
Observer is a new type of a NameNode in addition to Active and Standby in HA
settings. Observer Node maintains a replica of the namespace same as a Standby
Node. It additionally allows execution of clients read requests.
To ensure read-after-write consistency within a single client, a state ID is
introduced in RPC headers. The Observer responds to the client request only
after its own state has caught up with the client’s state ID, which it
previously received from the Active NameNode.
Clients can explicitly invoke a new client protocol call msync(), which ensures
that subsequent reads by this client from an Observer are consistent.
A new client-side ObserverReadProxyProvider is introduced to provide automatic
switching between Active and Observer NameNodes for submitting respectively
write and read requests.
> Consistent Reads from Standby Node
> ----------------------------------
>
> Key: HDFS-12943
> URL: https://issues.apache.org/jira/browse/HDFS-12943
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Major
> Fix For: 2.10.0, 3.3.0, 3.1.4, 3.2.2
>
> Attachments: ConsistentReadsFromStandbyNode.pdf,
> ConsistentReadsFromStandbyNode.pdf, HDFS-12943-001.patch,
> HDFS-12943-002.patch, HDFS-12943-003.patch, HDFS-12943-004.patch,
> TestPlan-ConsistentReadsFromStandbyNode.pdf
>
>
> StandbyNode in HDFS is a replica of the active NameNode. The states of the
> NameNodes are coordinated via the journal. It is natural to consider
> StandbyNode as a read-only replica. As with any replicated distributed system
> the problem of stale reads should be resolved. Our main goal is to provide
> reads from standby in a consistent way in order to enable a wide range of
> existing applications running on top of HDFS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
