[ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417625#comment-16417625 ]
Eric Yang commented on YARN-7935: --------------------------------- [~shaneku...@gmail.com] {quote} Eric Yang this isn't true for overlay networks. You can't assume Registry DNS will be in use and it won't be used by some of these network types without additional modifications to Hadoop (--dns for docker run). {quote} The information is almost straight out of [Docker embedded DNS|https://docs.docker.com/v17.09/engine/userguide/networking/configure-dns/]. I can be over concerned when developers get into a feature that they have not learn the basics. I am not blocking user-defined network feature to be implemented. As matter of the fact, I welcome to support user-defined network feature. However, calling Spark out as requiring user-defined network may not be the right message because I know it does not depend on user-defined network. I have previously implemented Spark 2.1 on docker without encountering the "limitation" of having to base on embedded DNS. This is the reason that it raised my eye balls on this issue to understand the technical detail and motivation on the attempts. Now I have the understanding of the motivation, code review will go much quicker. Back to the code review, the host can contain multiple network cards, using InetAddress.getLocalHost().getHostName() may not always produce desired result. It would be best to lookup Hadoop configuration to determine if there is specific hostname used (yarn.nodemanager.address) and do a address lookup if it is not 0.0.0.0, if the property doesn't exist, then fall back to use InetAddress.getLocalHost().getHostName(). > Expose container's hostname to applications running within the docker > container > ------------------------------------------------------------------------------- > > Key: YARN-7935 > URL: https://issues.apache.org/jira/browse/YARN-7935 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Priority: Major > Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch > > > Some applications have a need to bind to the container's hostname (like > Spark) which is different from the NodeManager's hostname(NM_HOST which is > available as an env during container launch) when launched through Docker > runtime. The container's hostname can be exposed to applications via an env > CONTAINER_HOSTNAME. Another potential candidate is the container's IP but > this can be addressed in a separate jira. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org