[
https://issues.apache.org/jira/browse/HDFS-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17572188#comment-17572188
]
Simbarashe Dzinamarira commented on HDFS-13522:
-----------------------------------------------
I've updated my pull requests to implement Design A+B. (This is built on
[~zhengzhuobinzzb]'s patch)
1) [IPC changes in hadoop-common|https://github.com/apache/hadoop/pull/4311]
2) [Propagating federated state when there are a few
namespaces.|https://github.com/apache/hadoop/pull/4584]
3) [Router changes to redirect reads to
observer.|https://github.com/apache/hadoop/pull/4127]
Only a [small
change|https://github.com/apache/hadoop/pull/4127/commits/37c7d9c60f803e347b9a33a1951159147e23096a]
was needed to implement Design B's behavior, by adding a new parameter
{_}dfs.federation.router.observer.federated.state.propagation.maxsize{_}.
I've broken down the overall implementation into the chained pull requests
above to make reviews easier. Hopefully that is an okay level of modularity.
> RBF: Support observer node from Router-Based Federation
> -------------------------------------------------------
>
> Key: HDFS-13522
> URL: https://issues.apache.org/jira/browse/HDFS-13522
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: federation, namenode
> Reporter: Erik Krogen
> Assignee: Simbarashe Dzinamarira
> Priority: Major
> Labels: pull-request-available
> Attachments: HDFS-13522.001.patch, HDFS-13522.002.patch,
> HDFS-13522_WIP.patch, HDFS-13522_proposal_zhengchenyu_v1.pdf, RBF_ Observer
> support.pdf, Router+Observer RPC clogging.png,
> ShortTerm-Routers+Observer.png,
> observer_reads_in_rbf_proposal_simbadzina_v1.pdf,
> observer_reads_in_rbf_proposal_simbadzina_v2.pdf
>
> Time Spent: 20h 10m
> Remaining Estimate: 0h
>
> Changes will need to occur to the router to support the new observer node.
> One such change will be to make the router understand the observer state,
> e.g. {{FederationNamenodeServiceState}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
