[
https://issues.apache.org/jira/browse/HDFS-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16827244#comment-16827244
]
Ayush Saxena commented on HDFS-13522:
-------------------------------------
Thanx Everyone for the discussion here. Feel like this three challenges.
[~elgoiri] already has mentioned two challenges :
* Collecting Observer state
* Invoking at the observer
* Handling the state id
The above two seems fairly straightforward. The main challenge seems to be
handling the state id. In a non federation scenario. A client gets the state id
for every operation at the Active and the client uses that id while invoking
the call at observer, which the observer uses to ensure non stale read.
The problem at RBF I feel is Router is mounted to different namespaces and a
client call can go to any of the namespace depending on the mount mapping.
So, the challenge may be handling the state id. That too may have two
approaches,that I can think of:
First we store the state id at the Router end and decide observer read at
Router making the client independent, For each call we check the state id
corresponding to the NS and invoke the call accordingly.
Second is what I think might be create a Router State which can be sent to the
Client, as is sent by the NN presently, and that may be decoded back to get
each of the namespace states, which can be used further.
The first one seems quite easy but major problem which I feel would be to sync
the state amongst all routers and the overhead that it will cause during an
operation, We have to read every time the value in this case from StateStore
and update the value everytime on write(may be a point of bottleneck too) and
with second the mechanism to wrap the state id stays a challange.
> Support observer node from Router-Based Federation
> --------------------------------------------------
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
> Reporter: Erik Krogen
> Assignee: Chao Sun
> Priority: Major
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
