[
https://issues.apache.org/jira/browse/HDFS-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349904#comment-16349904
]
Yiqun Lin commented on HDFS-13098:
----------------------------------
Looks very interesting, :)!
Only one question from me: Do these subclusters also dynamically generated?
That is say, we have N datanodes, then these datanodes communicate with the
Router. Based on the assignment policies, the Router decides which subcluster
the datanode belong to.
I am thinking some use cases for this. For the new machines joining the
cluster, it looks easy to deal with. But if the machines are moving across
subclusters, how we plan to do? Decommission in original subclsuter then add
into new subcluster?
> RBF: Datanodes interacting with Routers
> ---------------------------------------
>
> Key: HDFS-13098
> URL: https://issues.apache.org/jira/browse/HDFS-13098
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Íñigo Goiri
> Priority: Major
>
> Datanodes talk to particular Namenodes. We could use the Router
> infrastructure for the Datanodes to register/heartbeating into them and the
> Routers would forward this to particular Namenodes. This would make the
> assignment of Datanodes to subclusters potentially more dynamic.
> The implementation would potentially include:
> * Router to implement part of DatanodeProtocol
> * Forwarding DN messages into Routers
> * Policies to assign datanodes to subclusters
> * Datanodes to make blockpool configuration dynamic
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
