[jira] [Comment Edited] (YARN-7935) Expose container's hostname to applications running within the docker container
[ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16639019#comment-16639019 ] Suma Shivaprasad edited comment on YARN-7935 at 10/4/18 11:14 PM: -- This is no longer needed since the container's hostname, IP and additional information could be obtained via YARN-8569 by Spark or other AMs which need this information. was (Author: suma.shivaprasad): This is no longer needed since the container's hostname, IP and additional information could be obtained via YARN-8659 by Spark or other AMs which need this information. > Expose container's hostname to applications running within the docker > container > --- > > Key: YARN-7935 > URL: https://issues.apache.org/jira/browse/YARN-7935 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Major > Labels: Docker > Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch, > YARN-7935.4.patch > > > Some applications have a need to bind to the container's hostname (like > Spark) which is different from the NodeManager's hostname(NM_HOST which is > available as an env during container launch) when launched through Docker > runtime. The container's hostname can be exposed to applications via an env > CONTAINER_HOSTNAME. Another potential candidate is the container's IP but > this can be addressed in a separate jira. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7935) Expose container's hostname to applications running within the docker container
[
https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417175#comment-16417175
]
Shane Kumpf edited comment on YARN-7935 at 3/28/18 11:10 AM:
-
{quote}Docker embedded DNS will use /etc/resolv.conf from host, and filter out
local IP addresses (127.0.0.1 etc), if no entires are available, it will route
to 8.8.8.8
{quote}
[~eyang] this isn't true for overlay networks. You can't assume Registry DNS
will be in use and it won't be used by some of these network types without
additional modifications to Hadoop ({{--dns}} for {{docker run}}).
{quote}I am concerned that some end user code will end up invoking InetAddress
Java class{quote}
This will use the IP of the container and whatever resolver the container is
configured to use. Adding this environment variable doesn't change that.
I'm not seeing the issue with adding an additional environment variable that is
set to the same value as {{\-\-hostname}} if this solves a problem for a class
of application. No one is proposing modifying Hadoop IPC code to support NAT
here or to use the {{--link}} feature, just adding an additional environment
variable in non-entrypoint mode. Can you elaborate on the exact issue you see
this new environment variable causing?
was (Author: shaneku...@gmail.com):
{quote}Docker embedded DNS will use /etc/resolv.conf from host, and filter out
local IP addresses (127.0.0.1 etc), if no entires are available, it will route
to 8.8.8.8
{quote}
[~eyang] this isn't true for overlay networks. You can't assume Registry DNS
will be in use and it won't be used by some of these network types without
additional modifications to Hadoop ({{--dns}} for {{docker run}}).
{quote}I am concerned that some end user code will end up invoking InetAddress
Java class{quote}
This will use the IP of the container and whatever resolver the container is
configured to use. Adding this environment variable doesn't change that.
I'm not seeing the issue with adding an additional environment variable that is
set to the same value as {{--hostname}} if this solves a problem for a class of
application. No one is proposing modifying Hadoop IPC code to support NAT here
or to use the {{--link}} feature, just adding an additional environment
variable in non-entrypoint mode. Can you elaborate on the exact issue you see
this new environment variable causing?
> Expose container's hostname to applications running within the docker
> container
> ---
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like
> Spark) which is different from the NodeManager's hostname(NM_HOST which is
> available as an env during container launch) when launched through Docker
> runtime. The container's hostname can be exposed to applications via an env
> CONTAINER_HOSTNAME. Another potential candidate is the container's IP but
> this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7935) Expose container's hostname to applications running within the docker container
[
https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417175#comment-16417175
]
Shane Kumpf edited comment on YARN-7935 at 3/28/18 11:10 AM:
-
{quote}Docker embedded DNS will use /etc/resolv.conf from host, and filter out
local IP addresses (127.0.0.1 etc), if no entires are available, it will route
to 8.8.8.8
{quote}
[~eyang] this isn't true for overlay networks. You can't assume Registry DNS
will be in use and it won't be used by some of these network types without
additional modifications to Hadoop ({{--dns}} for {{docker run}}).
{quote}I am concerned that some end user code will end up invoking InetAddress
Java class{quote}
This will use the IP of the container and whatever resolver the container is
configured to use. Adding this environment variable doesn't change that.
I'm not seeing the issue with adding an additional environment variable that is
set to the same value as {{--hostname}} if this solves a problem for a class of
application. No one is proposing modifying Hadoop IPC code to support NAT here
or to use the {{--link}} feature, just adding an additional environment
variable in non-entrypoint mode. Can you elaborate on the exact issue you see
this new environment variable causing?
was (Author: shaneku...@gmail.com):
{quote}Docker embedded DNS will use /etc/resolv.conf from host, and filter out
local IP addresses (127.0.0.1 etc), if no entires are available, it will route
to 8.8.8.8
{quote}
[~eyang] this isn't true for overlay networks. You can't assume Registry DNS
will be in use and it won't be used by some of these network types without
additional modifications to Hadoop ({{--dns}} for {{docker run}}).
{quote}I am concerned that some end user code will end up invoking InetAddress
Java class{quote}
This will use the IP of the container and whatever resolver the container is
configured to use. Adding this environment variable doesn't change that.
I'm not seeing the issue with adding an additional environment variable that is
set to the same value as --hostname if this solves a problem for a class of
application. No one is proposing modifying Hadoop IPC code to support NAT here
or to use the {{--link}} feature, just adding an additional environment
variable in non-entrypoint mode. Can you elaborate on the exact issue you see
this new environment variable causing?
> Expose container's hostname to applications running within the docker
> container
> ---
>
> Key: YARN-7935
> URL: https://issues.apache.org/jira/browse/YARN-7935
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Major
> Attachments: YARN-7935.1.patch, YARN-7935.2.patch, YARN-7935.3.patch
>
>
> Some applications have a need to bind to the container's hostname (like
> Spark) which is different from the NodeManager's hostname(NM_HOST which is
> available as an env during container launch) when launched through Docker
> runtime. The container's hostname can be exposed to applications via an env
> CONTAINER_HOSTNAME. Another potential candidate is the container's IP but
> this can be addressed in a separate jira.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-7935) Expose container's hostname to applications running within the docker container
[ https://issues.apache.org/jira/browse/YARN-7935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16372322#comment-16372322 ] Suma Shivaprasad edited comment on YARN-7935 at 2/22/18 2:08 AM: - The current patch does not take care of "host" mode networking where the container's hostname should be set to the NM host's hostname. Will fix that and upload a patch. was (Author: suma.shivaprasad): The current patch does not take care of "host" mode networking where the container's hostname is set to the NM host's hostname. Will fix that and upload a patch. > Expose container's hostname to applications running within the docker > container > --- > > Key: YARN-7935 > URL: https://issues.apache.org/jira/browse/YARN-7935 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Major > Attachments: YARN-7935.1.patch, YARN-7935.2.patch > > > Some applications have a need to bind to the container's hostname (like > Spark) which is different from the NodeManager's hostname(NM_HOST which is > available as an env during container launch) when launched through Docker > runtime. The container's hostname can be exposed to applications via an env > CONTAINER_HOSTNAME. Another potential candidate is the container's IP but > this can be addressed in a separate jira. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
