[
https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417625#comment-16417625
]
Eric Yang commented on YARN-7935:
---------------------------------
[[email protected]] {quote}
Eric Yang this isn't true for overlay networks. You can't assume Registry DNS
will be in use and it won't be used by some of these network types without
additional modifications to Hadoop (--dns for docker run).
{quote}
The information is almost straight out of [Docker embedded
DNS|https://docs.docker.com/v17.09/engine/userguide/networking/configure-dns/].
I can be over concerned when developers get into a feature that they have not
learn the basics. I am not blocking user-defined network feature to be
implemented. As matter of the fact, I welcome to support user-defined network
feature. However, calling Spark out as requiring user-defined network may not
be the right message because I know it does not depend on user-defined network.
I have previously implemented Spark 2.1 on docker without encountering the
"limitation" of having to base on embedded DNS. This is the reason that it
raised my eye balls on this issue to understand the technical detail and
motivation on the attempts. Now I have the understanding of the motivation,
code review will go much quicker.
Back to the code review, the host can contain multiple network cards, using
InetAddress.getLocalHost().getHostName() may not always produce desired result.
It would be best to lookup Hadoop configuration to determine if there is
specific hostname used (yarn.nodemanager.address) and do a address lookup if it
is not 0.0.0.0, if the property doesn't exist, then fall back to use
InetAddress.getLocalHost().getHostName().
> Expose container's hostname to applications running within the docker
> container
> -------------------------------------------------------------------------------
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like
> Spark) which is different from the NodeManager's hostname(NM_HOST which is
> available as an env during container launch) when launched through Docker
> runtime. The container's hostname can be exposed to applications via an env
> CONTAINER_HOSTNAME. Another potential candidate is the container's IP but
> this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
