[
https://issues.apache.org/jira/browse/YARN-8675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16589066#comment-16589066
]
Eric Yang commented on YARN-8675:
---------------------------------
Further analysis of the problem indicated that the registry DNS is set to
mycluster.com, while the host level domain is example.com. If the hostname is
host1.example.com, spark on yarn cluster workload will start the container as
host1.mycluster.com. Host1.mycluster.com is unresolvable because no
registryDNS entry is written to zookeeper. Without using YARN service API,
there is no AM logic that handles registration of hostname to IP mapping. This
is the reason that it failed. For handling net=host situation properly without
using YARN service API, the registry DNS must set to same as host level domain,
which is example.com. System administrator must configure registryDNS domain
properly to permit application to use host level domain. This is to ensure
that decouple of infrastructure cluster (YARN), and workload cluster (YARN
apps). The application does not try to impersonate infrastructure cluster
unless explicitly allowed. This is a feature, not a bug.
> Setting hostname of docker container breaks with "host" networking mode for
> Apps which do not run as a YARN service
> -------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-8675
> URL: https://issues.apache.org/jira/browse/YARN-8675
> Project: Hadoop YARN
> Issue Type: Bug
> Reporter: Yesha Vora
> Assignee: Suma Shivaprasad
> Priority: Major
> Labels: Docker
>
> Applications like the Spark AM currently do not run as a YARN service and
> setting hostname breaks driver/executor communication if docker version
> >=1.13.1 , especially with wire-encryption turned on.
> YARN-8027 sets the hostname if YARN DNS is enabled. But the cluster could
> have a mix of YARN service/native Applications.
> The proposal is to not set the hostname when "host" networking mode is
> enabled.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
