[
https://issues.apache.org/jira/browse/HADOOP-15774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622622#comment-16622622
]
Íñigo Goiri commented on HADOOP-15774:
--------------------------------------
[[email protected]], I've been reading the documentation and the code for the
YARN registry and I think it covers the storage part (including the API for
registering and discoverying) but it would need to be integrated with the
components.
I think it would make sense to still modify the following components:
* Namenode: register itself in the registry at startup time.
* Datanode: use the YARN Registry API for discovery and setup the blockpool
based on this.
* Resource Manager: register itself in the registry at startup time.
* Node Manager: use the YARN Registry API for discovery and tune the client to
connect to the proper RMs.
I think it still makes sense to have an intermediate class to switch between
the current XML Configuration approach and this new service.
A couple limitation I see in the current registry:
* Would it make sense to make the registry a standalone service instead of
running in the RM?
* Being part of the YARN may bring some weird dependencies (HDFS->YARN), any
ideas here? Should it be move to a separate place? Maybe parts?
We should also rename this effort to something like "Leveraging YARN register
in Hadoop services".
Thoughts?
We can refocus the work in that direction if so.
> Discovery of HA servers
> -----------------------
>
> Key: HADOOP-15774
> URL: https://issues.apache.org/jira/browse/HADOOP-15774
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Íñigo Goiri
> Priority: Major
> Attachments: Discovery Service.pdf
>
>
> Currently, Hadoop relies on configuration files to specify the servers.
> This requires maintaining these configuration files and propagating the
> changes.
> Hadoop should have a framework to provide discovery.
> For example, in HDFS, we could define the Namenodes in a shared location and
> the DNs would use the framework to find the Namenodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
