[
https://issues.apache.org/jira/browse/MESOS-3711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vasilis Vasaitis updated MESOS-3711:
------------------------------------
Description:
(Disclaimer: I'm not the one running the Mesos infrastructure in my org, and I
don't necessarily fully understand how all the moving parts fit together, so
please bear with me if there any gaps in my understanding of the issues at
hand.)
We have a setup here where we deploy Docker-based tasks on Mesos, using Aurora
(and thus the Thermos executor, on the agent side). As part of the process of
launching a task, it looks like the Mesos agent creates / volume-mounts an
{{/mnt/mesos/sandbox}} directory, which is what's used as the task's sandbox.
Thermos then creates a {{sandbox}} subdirectory _inside_ that, and the
aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the directory that the user
application is given. So far so good.
Now, Docker has the option, during the creation of a Docker image, to specify
the _user_ that any container launched using this image will be run as. This is
a useful feature, because often the image is built so that only one particular
user has ownership of important files etc. One could of course sidestep this
issue by always launching the container as root, but that can be unsavoury for
its own reasons.
However, with the setup I described above, specifying a user for the Docker
container quickly goes south: the Thermos executor itself is launched as that
user, tries to create that extra {{sandbox}} directory, and fails, because the
parent directory is owned by root.
I won't claim to know whether this is the _best_ approach, but one possible
solution to this problem is to chmod 1777 the parent sandbox directory (i.e.,
set the sticky bit, like {{/tmp}}) after creating it; this way any user can
create files/directories under it, without compromising the isolation between
users.
was:
(Disclaimer: I'm not the one running the Mesos infrastructure in my org, and I
don't necessarily fully understand how all the moving parts fit together, so
please bear with me if there any gaps in my understanding of the issues at
hand.)
We have a setup here where we deploy Docker-based tasks on Mesos, using Aurora
(and thus the Thermos executor, on the agent side). As part of the process of
launching a task, it looks like the Mesos agent creates / volume-mounts an
{{/mnt/mesos/sandbox}} directory, which is what's used as the task's sandbox.
Thermos then creates a {{sandbox}} subdirectory _inside_ that, and the
aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the directory that the user
application is given. So far so good.
Now, Docker has the option, during the creation of a Docker image, to specify
the _user_ that any container launched using this image will be run as. This is
a useful feature, because often the image is built so that only one particular
user has ownership of important files etc. One could of course sidestep this
issue by always launching the container as root, but that can be unsavoury for
its own reasons.
However, with the setup I described above, specifying a user for the Docker
container quickly goes south: the Thermos executor itself is launched as that
user, tries to create that extra {{sandbox}} directory, and fails, because the
parent directory is owned by root.
I won't claim to know whether this is the _best_ approach, but one possible
solution to this problem is to chmod 1777 the parent sandbox directory (i.e.,
set the sticky bit, like {{/tmp}}) after creating it; this way any user can
create files/directories under it, without compromising the isolation between
users. I plan to attach a small patch which demonstrates this change.
> Docker containers running as other than root can't access sandbox
> -----------------------------------------------------------------
>
> Key: MESOS-3711
> URL: https://issues.apache.org/jira/browse/MESOS-3711
> Project: Mesos
> Issue Type: Bug
> Components: containerization, docker
> Reporter: Vasilis Vasaitis
>
> (Disclaimer: I'm not the one running the Mesos infrastructure in my org, and
> I don't necessarily fully understand how all the moving parts fit together,
> so please bear with me if there any gaps in my understanding of the issues at
> hand.)
> We have a setup here where we deploy Docker-based tasks on Mesos, using
> Aurora (and thus the Thermos executor, on the agent side). As part of the
> process of launching a task, it looks like the Mesos agent creates /
> volume-mounts an {{/mnt/mesos/sandbox}} directory, which is what's used as
> the task's sandbox. Thermos then creates a {{sandbox}} subdirectory _inside_
> that, and the aggregate {{/mnt/mesos/sandbox/sandbox}} is in fact the
> directory that the user application is given. So far so good.
> Now, Docker has the option, during the creation of a Docker image, to specify
> the _user_ that any container launched using this image will be run as. This
> is a useful feature, because often the image is built so that only one
> particular user has ownership of important files etc. One could of course
> sidestep this issue by always launching the container as root, but that can
> be unsavoury for its own reasons.
> However, with the setup I described above, specifying a user for the Docker
> container quickly goes south: the Thermos executor itself is launched as that
> user, tries to create that extra {{sandbox}} directory, and fails, because
> the parent directory is owned by root.
> I won't claim to know whether this is the _best_ approach, but one possible
> solution to this problem is to chmod 1777 the parent sandbox directory (i.e.,
> set the sticky bit, like {{/tmp}}) after creating it; this way any user can
> create files/directories under it, without compromising the isolation between
> users.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
