[
https://issues.apache.org/jira/browse/HADOOP-15774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622335#comment-16622335
]
Íñigo Goiri commented on HADOOP-15774:
--------------------------------------
Thanks for the comments, some responses:
bq. Once server addresses are updated, how sever manager plan to do and let
workers known?
In the simple case, it would be the manager itself (e.g., Namenode) which would
register there.
Then for federation, that already happens with the heartbeat.
I will call this out in the next iteration of the document detailing the
interfaces to register.
bq. For the workers, do they need to get server address every time when they
ask for a request for servers? Or they will just get the address once and
cached it in itself?
It should be cache and we should have a -refresh call to reload the
configuration.
bq. The yarn registry is designed precisely for this kind of thing, and has DNS
integration
I have been listening about it but never went too deep as it had the YARN
prefix.
After reading the [latest
documentation|https://github.com/apache/hadoop/blob/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/markdown/registry/yarn-registry.md]
it looks like this is it.
[[email protected]], as you are familiar with the work, what would be missing
from the description in [^Discovery Service.pdf]? Integrating the DNs with
this? Is there any JIR tracking this? I see YARN-913 has a few JIRAs left but
none seem related.
I'll try to setup a cluster using the YARN Registry and I'll see if there are
gaps left.
> Discovery of HA servers
> -----------------------
>
> Key: HADOOP-15774
> URL: https://issues.apache.org/jira/browse/HADOOP-15774
> Project: Hadoop Common
> Issue Type: New Feature
> Reporter: Íñigo Goiri
> Priority: Major
> Attachments: Discovery Service.pdf
>
>
> Currently, Hadoop relies on configuration files to specify the servers.
> This requires maintaining these configuration files and propagating the
> changes.
> Hadoop should have a framework to provide discovery.
> For example, in HDFS, we could define the Namenodes in a shared location and
> the DNs would use the framework to find the Namenodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
