[
https://issues.apache.org/jira/browse/HDFS-13522?focusedWorklogId=782049&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-782049
]
ASF GitHub Bot logged work on HDFS-13522:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 16/Jun/22 13:37
Start Date: 16/Jun/22 13:37
Worklog Time Spent: 10m
Work Description: zhengchenyu commented on PR #4441:
URL: https://github.com/apache/hadoop/pull/4441#issuecomment-1157672472
> Thanks @zhengchenyu and @simbadzina .
>
> > I think config in client side may be more flexible.
>
> This is a very meaningful topic. If only the client controls whether or
not to enable ObserverRead will be more difficult for Admin to control, because
it is very difficult to upgrade the HDFS client in full. In other words: If RBF
controls whether the ObserverRead is enabled, the Admin will be very convenient
to control the ObserverRead of the entire cluster, and even dynamically control
whether the ObserverRead of a single NS or the entire cluster is enabled. But
there may be some special Client that do not want to enable ObserverRead, so
RBF should identify those requests and proxy them to the Active Namenode.
>
> @simbadzina This is why dynamic updates are required, so that when Admin
finds that there are some abnormal Observer NameNodes, he/she can quickly
disable the ObserverRead of one NS or even all NSs.
>
> > In our draft design, after apply
[HDFS-13522](https://issues.apache.org/jira/browse/HDFS-13522).002.patch, I
wanna proxy client's state id.
>
> Proxying client's state id to the NameNode by RBF will be very complicated.
>
> * A DFSClient may read or write some paths of different NameServices, and
the stateID of different NS may be different.
> * The client does not know the Nameservice to which the reading or writing
path belong, so it cannot pass the state id to RBF.
Yes, you are right in some condition. If all client are common user, for
hive and mr application, it is right.
We know observer can not guarantee strong consistency, maybe some use have
high demand, they could wanna disable observe read, though few user have this
demand.
Maybe we can reserve configuration both on router side and client side.
Yes, Proxying client's state id is complicated. I don't know whether it is
necessary or not. So just delay it.
Issue Time Tracking
-------------------
Worklog Id: (was: 782049)
Time Spent: 13.5h (was: 13h 20m)
> RBF: Support observer node from Router-Based Federation
> -------------------------------------------------------
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
> Reporter: Erik Krogen
> Assignee: Simbarashe Dzinamarira
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch,
> HDFS-13522_WIP.patch, RBF_ Observer support.pdf, Router+Observer RPC
> clogging.png, ShortTerm-Routers+Observer.png
>
> Time Spent: 13.5h
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
