Re: Mesos containerizer & isolation

[Jie Yu]

To add to haosdent's reply:

- I have a USER directive in my Dockerfile in order for the CMD to be
> executed as that user, but that does not seem to be supported (yet?) by the
> Docker image provider. Is there any method (except `sudo`/`setuser`) to
> achieve running as a user present in the image's /etc/fstab?


Currently, USER directive in Dockerfile is not honored. You can think of
that as using `-u` when doing docker run, and uses the uid of the 'user' on
the host ('user' here is what's specified in CommandInfo.user or
frameworkInfo.user if the former is not specified). The reason we need to
do that is because we want to make sure the processes in the container can
access its sandbox and persistent volumes which is owned by 'user'.

This can be potentially solved by using user namespace as haosdent pointed
out.

- I may have to run untrusted code, so can I make sure that users cannot
break out of the chroot? What about UID namespacing, so that root in the
chroot does not become root on the host system when breaking out?

You can run your code using an unprivileged user (e.g., nobody). You just
need to set CommandInfo.user.

- Jie

On Wed, Nov 2, 2016 at 7:14 PM, haosdent  wrote:

> >- Is it possible to hide host processes from the container?
> You may consider to use the namespaces/pid isolator, add `namespaces/pid`
> in the `--isolation` flag when launch Mesos Agent
> > -Is it possible to run processes that open network ports (possibly
> already open on the host system) and have them mapped to different ports on
> the host system, just as with Docker's `-p`?
> You need to use CNI port mapping. Refer to its document
> https://reviews.apache.org/r/53015/
> >  Is there any method (except `sudo`/`setuser`) to achieve running as a
> user present in the image's /etc/fstab?
> Mesos don't support user namespace now, need to use su to switch users
>
> On Thu, Nov 3, 2016 at 9:56 AM, Tobias Pfeiffer  wrote:
>
>> Actually, say I was in a fancy mood, could I actually *not* use the
>> Docker image provider and instead run `nvidia-docker run [more hand-crafted
>> parameters] myimage ` as an ordinary command within the Mesos
>> container, or would I have to dig very deep into Mesos to find the right
>> parameters to pass to nvidia-docker?
>>
>> Thanks
>> Tobias
>>
>> On Thu, Nov 3, 2016 at 10:18 AM, Tobias Pfeiffer 
>> wrote:
>>
>>> Hi,
>>>
>>> I asked this question also yesterday in the #mesos channel on IRC, but I
>>> guess due to timezone differences there were not many people awake and/or
>>> working, sorry for reposting. (Maybe someone answered after I left, but it
>>> seems that the IRC bot is only archiving channel joins/leaves? ->
>>> http://wilderness.apache.org/channels/?f=apache-syncope/2016-11-02)
>>>
>>> My question is about the Mesos containerizer. I want to run code using
>>> the Mesos GPU support and the docs state that this is currently only
>>> supported by the Mesos containerizer. So my understanding of using the
>>> Mesos containerizer with Docker images is that
>>> - the content of the Docker images is unpacked to the filesystem (using
>>> one of the provisioner backends, such as "copy" or "overlay")
>>> - the user's command is executed in a chroot in that directory.
>>> Is that correct?
>>>
>>> The first thing I noticed is (besides a much higher latency due to the
>>> image provisioning process) that `ps aux` and `hostname` expose details of
>>> the host system, so I was wondering about the level of isolation that I can
>>> achieve with the Mesos containerizer, as opposed to running in a Docker
>>> container. In particular:
>>> - Is it possible to hide host processes from the container?
>>> - Is it possible to run processes that open network ports (possibly
>>> already open on the host system) and have them mapped to different ports on
>>> the host system, just as with Docker's `-p`?
>>> - I have a USER directive in my Dockerfile in order for the CMD to be
>>> executed as that user, but that does not seem to be supported (yet?) by the
>>> Docker image provider. Is there any method (except `sudo`/`setuser`) to
>>> achieve running as a user present in the image's /etc/fstab?
>>> - I may have to run untrusted code, so can I make sure that users cannot
>>> break out of the chroot? What about UID namespacing, so that root in the
>>> chroot does not become root on the host system when breaking out?
>>>
>>> Thanks for your help
>>> Tobias
>>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>

Re: Mesos containerizer & isolation

[haosdent]

>- Is it possible to hide host processes from the container?
You may consider to use the namespaces/pid isolator, add `namespaces/pid`
in the `--isolation` flag when launch Mesos Agent
> -Is it possible to run processes that open network ports (possibly
already open on the host system) and have them mapped to different ports on
the host system, just as with Docker's `-p`?
You need to use CNI port mapping. Refer to its document
https://reviews.apache.org/r/53015/
>  Is there any method (except `sudo`/`setuser`) to achieve running as a
user present in the image's /etc/fstab?
Mesos don't support user namespace now, need to use su to switch users

On Thu, Nov 3, 2016 at 9:56 AM, Tobias Pfeiffer  wrote:

> Actually, say I was in a fancy mood, could I actually *not* use the Docker
> image provider and instead run `nvidia-docker run [more hand-crafted
> parameters] myimage ` as an ordinary command within the Mesos
> container, or would I have to dig very deep into Mesos to find the right
> parameters to pass to nvidia-docker?
>
> Thanks
> Tobias
>
> On Thu, Nov 3, 2016 at 10:18 AM, Tobias Pfeiffer  wrote:
>
>> Hi,
>>
>> I asked this question also yesterday in the #mesos channel on IRC, but I
>> guess due to timezone differences there were not many people awake and/or
>> working, sorry for reposting. (Maybe someone answered after I left, but it
>> seems that the IRC bot is only archiving channel joins/leaves? ->
>> http://wilderness.apache.org/channels/?f=apache-syncope/2016-11-02)
>>
>> My question is about the Mesos containerizer. I want to run code using
>> the Mesos GPU support and the docs state that this is currently only
>> supported by the Mesos containerizer. So my understanding of using the
>> Mesos containerizer with Docker images is that
>> - the content of the Docker images is unpacked to the filesystem (using
>> one of the provisioner backends, such as "copy" or "overlay")
>> - the user's command is executed in a chroot in that directory.
>> Is that correct?
>>
>> The first thing I noticed is (besides a much higher latency due to the
>> image provisioning process) that `ps aux` and `hostname` expose details of
>> the host system, so I was wondering about the level of isolation that I can
>> achieve with the Mesos containerizer, as opposed to running in a Docker
>> container. In particular:
>> - Is it possible to hide host processes from the container?
>> - Is it possible to run processes that open network ports (possibly
>> already open on the host system) and have them mapped to different ports on
>> the host system, just as with Docker's `-p`?
>> - I have a USER directive in my Dockerfile in order for the CMD to be
>> executed as that user, but that does not seem to be supported (yet?) by the
>> Docker image provider. Is there any method (except `sudo`/`setuser`) to
>> achieve running as a user present in the image's /etc/fstab?
>> - I may have to run untrusted code, so can I make sure that users cannot
>> break out of the chroot? What about UID namespacing, so that root in the
>> chroot does not become root on the host system when breaking out?
>>
>> Thanks for your help
>> Tobias
>>
>
>


-- 
Best Regards,
Haosdent Huang

Mesos containerizer & isolation

[Tobias Pfeiffer]

Hi,

I asked this question also yesterday in the #mesos channel on IRC, but I
guess due to timezone differences there were not many people awake and/or
working, sorry for reposting. (Maybe someone answered after I left, but it
seems that the IRC bot is only archiving channel joins/leaves? ->
http://wilderness.apache.org/channels/?f=apache-syncope/2016-11-02)

My question is about the Mesos containerizer. I want to run code using the
Mesos GPU support and the docs state that this is currently only supported
by the Mesos containerizer. So my understanding of using the Mesos
containerizer with Docker images is that
- the content of the Docker images is unpacked to the filesystem (using one
of the provisioner backends, such as "copy" or "overlay")
- the user's command is executed in a chroot in that directory.
Is that correct?

The first thing I noticed is (besides a much higher latency due to the
image provisioning process) that `ps aux` and `hostname` expose details of
the host system, so I was wondering about the level of isolation that I can
achieve with the Mesos containerizer, as opposed to running in a Docker
container. In particular:
- Is it possible to hide host processes from the container?
- Is it possible to run processes that open network ports (possibly already
open on the host system) and have them mapped to different ports on the
host system, just as with Docker's `-p`?
- I have a USER directive in my Dockerfile in order for the CMD to be
executed as that user, but that does not seem to be supported (yet?) by the
Docker image provider. Is there any method (except `sudo`/`setuser`) to
achieve running as a user present in the image's /etc/fstab?
- I may have to run untrusted code, so can I make sure that users cannot
break out of the chroot? What about UID namespacing, so that root in the
chroot does not become root on the host system when breaking out?

Thanks for your help
Tobias