[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16641048#comment-16641048
]
Till Toenshoff commented on MESOS-6340:
---
...and *again* ran into it, this time just like the trap Cody described - a
python based task trying to extract the home-dir of the user "nobody" which had
no home-dir entry in passwd. Is there any reason to not follow what
[~qianzhang] suggested?
/cc [~gilbert] [~jieyu]
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: agent, containerization
>Reporter: Cody Maloney
>Priority: Major
> Labels: containerizer
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16629784#comment-16629784
]
Till Toenshoff commented on MESOS-6340:
---
I ran into this challenge (trap?!) myself. The symptom was curl failing to work
as soon as SSL certificate validation was enabled with a rather nonsensical
"curl: (27) Out of memory}}. This happens due to curl failing to gain knowledge
of the home-dir of the user in my docker image.
Let's please improve this soon and / or document all the possible shortcomings,
sparing our users the debug sessions.
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: agent, containerization
>Reporter: Cody Maloney
>Priority: Major
> Labels: containerizer
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431642#comment-16431642
]
Qian Zhang commented on MESOS-6340:
---
I think our solution can be: If user does not set {{HOME}} explicitly, set it
to {{$MESOS_SANDBOX}} which is the value of agent's flag
{{--sandbox_directory}} for the container with image specified or the path to
the container sandbox on the agent host filesystem for the container without
image specified.
The reason that I do not suggest to get user's home directory from /etc/passwd
in the container image is, in Mesos containerizer, we always launch container
with the user on the agent host rather than the user in the container image
which is different from Docker. However this seems another area in Mesos
containerizer that we can improve: For the container with image specified, we
should change to use the user in the image to launch the container. [~jieyu]
and [~gilbert], how do you think?
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: agent, containerization
>Reporter: Cody Maloney
>Priority: Major
> Labels: containerizer
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16414880#comment-16414880
]
Qian Zhang commented on MESOS-6340:
---
I found Docker (actually it is {{runc}} internally) will always set {{HOME}}
env var when launching a container:
https://github.com/opencontainers/runc/blob/master/libcontainer/init_linux.go#L319:L324
Its logic is if user does not set {{HOME}} explicitly, get user’s home
directory from /etc/passwd and set {{HOME}} to it.
So user may see different behaviors when launching a container via Mesos
containerizer and Docker containerizer from the same Docker image, i.e., the
former will not have {{HOME}} set but the later will have, that could be a
problem, it might be a burden for customers to move from Docker containerizer
to UCR.
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: agent, containerization
>Reporter: Cody Maloney
>Priority: Major
> Labels: containerizer
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16408447#comment-16408447
]
michael beisiegel commented on MESOS-6340:
--
I'm using a docker image based on ubuntu:16.04 with vi installed. After 'dcos
task exec ...' into the container I get the following messages.
*on start vi*
{color:#205081}Cannot execute shell /usr/bin/bash{color}
{color:#205081}E79: Cannot expand wildcards{color}
{color:#205081}Cannot execute shell /usr/bin/bash{color}
{color:#205081}E79: Cannot expand wildcards{color}
*on close vi*
{color:#205081}E138: Can't write viminfo file $HOME/.viminfo! {color}
{color:#205081}Press ENTER or type command to continue{color}
The HOME env var is not set. Once I set HOME to /root or $MESOS_SANDBOX the
messages are gone.
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: agent, containerization
>Reporter: Cody Maloney
>Priority: Major
> Labels: containerizer
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15562768#comment-15562768
]
Joshua Cohen commented on MESOS-6340:
-
Yes, this is relatively accurate. Thermos currently has some pretty janky logic
to set {{$HOME}}, which amounts to:
* If running a task with an isolated filesystem (i.e. unified containerizer
with an {{Image}} set), then use {{/mnt/mesos/sandbox}} (which is bind mounted
to the rootfs for the isolated task filesystem).
* Otherwise use {{$MESOS_SANDBOX}}.
I think given the current implementation of filesystem isolation in Thermos,
the first bit of logic is pretty specific to your implementation though (we
don't set the filesystem {{Image}} on the {{ContainerInfo}}, instead we just
add it as a volume and Thermos uses {{mesos-containerizer launch}} to
pivot/chroot into that filesystem after the task is launched. This allows our
executor to rely on dependencies from the host filesystem (e.g. Python) without
requiring that all filesystem images include those dependencies.
Long story short, I think from Mesos's perspective, just setting {{$HOME}} to
be {{$MESOS_SANDBOX}} would be acceptable (but that's based on the assumption
that {{$MESOS_SANDBOX}} and {{$MESOS_DIRECTORY}} point to the same path when
not running under filesystem isolation.
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: containerization, slave
>Reporter: Cody Maloney
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MESOS-6340) Set HOME for Mesos tasks
[
https://issues.apache.org/jira/browse/MESOS-6340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15556516#comment-15556516
]
Zameer Manji commented on MESOS-6340:
-
Thermos (Aurora's executor) works around this issue by setting {{$HOME}} to cwd
{{$WORK_DIR}} and/or using {{$MESOS_SANDBOX}} when it is set.
I think [~joshua.cohen] can confirm or deny this.
Personally, if $HOME could default to those values that would be fantastic.
Executors can do their own customization if needed, but setting something would
be better than nothing.
> Set HOME for Mesos tasks
>
>
> Key: MESOS-6340
> URL: https://issues.apache.org/jira/browse/MESOS-6340
> Project: Mesos
> Issue Type: Bug
> Components: containerization, slave
>Reporter: Cody Maloney
>Assignee: Jie Yu
>
> Quite a few programs assume {{$HOME}} points to a user-editable data file
> directory.
> One example is PYTHON, which tries to look up $HOME to find user-installed
> pacakges, and if that fails it tries to look up the user in the passwd
> database which often goes badly (The container is running under the `nobody`
> user):
> {code}
> if i == 1:
> if 'HOME' not in os.environ:
> import pwd
> userhome = pwd.getpwuid(os.getuid()).pw_dir
> else:
> userhome = os.environ['HOME']
> {code}
> Just setting HOME by default to WORK_DIR would enable more software to work
> correctly out of the box. Software which needs to specialize / change it (or
> schedulers with specific preferences), should still be able to set it
> arbitrarily and anything a scheduler explicitly sets should overwrite the
> default value of $WORK_DIR
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
