[
https://issues.apache.org/jira/browse/YARN-4757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193577#comment-15193577
]
Robert Joseph Evans commented on YARN-4757:
-------------------------------------------
I am +1 on the idea of using DNS for long lived service discovery, but we need
to be very very careful about security. If we are not all of the problems
possible with https://en.wikipedia.org/wiki/DNS_spoofing would likely be
possible with this too. We need to be positive that we can restrict the names
allowed so there are no conflicts with other servers on the network/internet.
Additionally if we make this super simple, which is the entire goal here, then
we are covering up some really potentially serious issues with client code,
that a normal server running off YARN would not expect to have. It really
comes down to any service running on YARN that wants to be secure needs to have
2 way authentication client authenticates server and server authenticates
clients. There are timing attacks and other things that can happen when a
process crashes and lets go of a port. Internal web services especially feel
vulnerable because unless you enable SSL they will be insecure, something that
many groups avoid on internal services because of the extra overhead of doing
encryption.
Do you plan on handling ephemeral ports in some way? As far as I know there is
no standard for including port(s) in a DNS entry. If we do come up with
something that is non-standard doesn't that still necessitate client side
changes which was an expressed goal of this JIRA? If we don't handle ephemeral
ports are we going to add in mesos-like scheduling of ports?
> [Umbrella] Simplified discovery of services via DNS mechanisms
> --------------------------------------------------------------
>
> Key: YARN-4757
> URL: https://issues.apache.org/jira/browse/YARN-4757
> Project: Hadoop YARN
> Issue Type: New Feature
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Jonathan Maron
>
> [See overview doc at YARN-4692, copying the sub-section (3.2.10.2) to track
> all related efforts.]
> In addition to completing the present story of service-registry (YARN-913),
> we also need to simplify the access to the registry entries. The existing
> read mechanisms of the YARN Service Registry are currently limited to a
> registry specific (java) API and a REST interface. In practice, this makes it
> very difficult for wiring up existing clients and services. For e.g, dynamic
> configuration of dependent endpoints of a service is not easy to implement
> using the present registry-read mechanisms, *without* code-changes to
> existing services.
> A good solution to this is to expose the registry information through a more
> generic and widely used discovery mechanism: DNS. Service Discovery via DNS
> uses the well-known DNS interfaces to browse the network for services.
> YARN-913 in fact talked about such a DNS based mechanism but left it as a
> future task. (Task) Having the registry information exposed via DNS
> simplifies the life of services.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
