[
https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15293665#comment-15293665
]
Steve Loughran commented on YARN-4757:
--------------------------------------
I am supportive of this; we always had a goal of supporting DNS (somehow); our
naming policy was designed for that.
A troublespot is going to be that user names, even/especially kerberos names,
may not be valid for DNS. AD accounts often have spaces in; i18n names exist,
then there is punctuation. In YARN-913 I did try using punycode, but that only
converts high-ASCII, high-unicode chars down —it does nothing for spaces in a
name.
We can control the other bits, but not that.
Related to that, the parse guidelines on p8 shouldn't have a mixedcase example
"aUser", as that my be misconstrued as case being relevant.
p12; architecture good, just bear in mind that the DNS service will need to
handle ZK node failure/failover, so be able to switch to a new ZK node and
(presumably) rebuild its state from that one. Similarly, implementation will
need to handle the startup state 'zk server not yet live'.
I think you may need to add a sequence diagram for that; essentially re-enum
the tree with records updated to match the new state as appropriate
h3. implementation (p22).
* I think you may want to use one of the guava caches for doing some caching
here, it could make a big diff to ZK load in some scenarios. Example:
bootstrapping an app across the cluster which looks up the HBase record.
* I just had a look at how Antonio Lain did the DNS binding in Smartfrog, where
the Anubis HA T-Space was used as a P2P equivalent to ZK:
https://sourceforge.net/p/smartfrog/svn/HEAD/tree/trunk/core/components/dns/src/org/smartfrog/services/dns/
. He used dnsjava, though also added the option to actually bring up BIND:
https://sourceforge.net/p/smartfrog/svn/HEAD/tree/trunk/core/components/dns/src/org/smartfrog/services/dns/DNSBindNamedImpl.java
I'd go with dnsjava.
Note I also wrote a prototype REST server for the registry; that's hidden in
slider but it could be copied over to {{yarn-registry}}; it helped find some
JSON/jackson marshalling problems with field naming already. The Java registry
API was all built on the option of going RESTy later, and it would be nice for
going through Knox for remote access.
Allan: consul gets a mention in the docs. Essentially, the YARN service
registry works as the repo, all that's needed is to serve this up as a
different protocol through 1+ node. Because of the ZK option, code which works
directly with ZK can work with the data without worrying about DNS, DNS caching
and server failures.
> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
> Key: YARN-4757
> URL: https://issues.apache.org/jira/browse/YARN-4757
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Jonathan Maron
> Attachments:
> 0001-YARN-4757-Initial-code-submission-for-DNS-Service.patch, YARN-4757-
> Simplified discovery of services via DNS mechanisms.pdf
>
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to track
> all related efforts.]
> In addition to completing the present story of service-registry (YARN-913),
> we also need to simplify the access to the registry entries. The existing
> read mechanisms of the YARN Service Registry are currently limited to a
> registry specific (java) API and a REST interface. In practice, this makes it
> very difficult for wiring up existing clients and services. For e.g, dynamic
> configuration of dependent endpoints of a service is not easy to implement
> using the present registry-read mechanisms, *without* code-changes to
> existing services.
> A good solution to this is to expose the registry information through a more
> generic and widely used discovery mechanism: DNS. Service Discovery via DNS
> uses the well-known DNS interfaces to browse the network for services.
> YARN-913 in fact talked about such a DNS based mechanism but left it as a
> future task. (Task) Having the registry information exposed via DNS
> simplifies the life of services.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
