[
https://issues.apache.org/jira/browse/MESOS-5307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jie Yu updated MESOS-5307:
--------------------------
Description:
Currently, if a container uses container image, we'll do a bind mount of its
sandbox (<sandbox> -> <rootfs>/mnt/mesos/sandbox) in the host mount namespace.
However, doing the mounts in the host mount table is not ideal. That
complicates both the cleanup path and the recovery path.
Instead, we can do the sandbox bind mount in the container's mount namespace so
that cleanup and recovery will be greatly simplified. We can setup mount
propagation properly so that persistent volumes mounted at <sandbox>/xxx can be
propagated into the container.
Here is a simple proof of concept:
Console 1:
{noformat}
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ ll .
total 12
drwxrwxr-x 3 vagrant vagrant 4096 Apr 25 16:05 ./
drwxrwxr-x 6 vagrant vagrant 4096 Apr 25 23:17 ../
drwxrwxr-x 5 vagrant vagrant 4096 Apr 25 23:17 slave/
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ ll slave/
total 20
drwxrwxr-x 5 vagrant vagrant 4096 Apr 25 23:17 ./
drwxrwxr-x 3 vagrant vagrant 4096 Apr 25 16:05 ../
drwxrwxr-x 6 vagrant vagrant 4096 Apr 26 21:06 directory/
drwxr-xr-x 12 vagrant vagrant 4096 Apr 25 23:20 rootfs/
drwxrwxr-x 2 vagrant vagrant 4096 Apr 25 16:09 volume/
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ sudo mount --bind slave/ slave/
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ sudo mount --make-shared slave/
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ cat /proc/self/mountinfo
50 22 8:1 /home/vagrant/tmp/mesos/slave /home/vagrant/tmp/mesos/slave
rw,relatime shared:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
{noformat}
Console 2:
{noformat}
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ cd slave/
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave$ sudo unshare -m /bin/bash
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# sudo mount --make-rslave .
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# cat /proc/self/mountinfo
124 63 8:1 /home/vagrant/tmp/mesos/slave /home/vagrant/tmp/mesos/slave
rw,relatime master:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# mount --rbind directory/
rootfs/mnt/mesos/sandbox/
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# mount --rbind rootfs/ rootfs/
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# mount -t proc proc rootfs/proc
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# pivot_root rootfs
rootfs/tmp/.rootfs
root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# cd /
root@vagrant-ubuntu-trusty-64:/# cat /proc/self/mountinfo
126 61 8:1 /home/vagrant/tmp/mesos/slave/rootfs / rw,relatime master:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
127 126 8:1 /home/vagrant/tmp/mesos/slave/directory /mnt/mesos/sandbox
rw,relatime master:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
128 126 0:3 / /proc rw,relatime - proc proc rw
{noformat}
Console 1:
{noformat}
agrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ cd slave/
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave$ sudo mount --bind volume/
directory/v1
vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave$ cat /proc/self/mountinfo
50 22 8:1 /home/vagrant/tmp/mesos/slave /home/vagrant/tmp/mesos/slave
rw,relatime shared:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
129 50 8:1 /home/vagrant/tmp/mesos/slave/volume
/home/vagrant/tmp/mesos/slave/directory/v1 rw,relatime shared:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
{noformat}
Console 2:
{noformat}
root@vagrant-ubuntu-trusty-64:/# cat /proc/self/mountinfo
126 61 8:1 /home/vagrant/tmp/mesos/slave/rootfs / rw,relatime master:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
127 126 8:1 /home/vagrant/tmp/mesos/slave/directory /mnt/mesos/sandbox
rw,relatime master:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
128 126 0:3 / /proc rw,relatime - proc proc rw
132 127 8:1 /home/vagrant/tmp/mesos/slave/volume /mnt/mesos/sandbox/v1
rw,relatime shared:4 master:1 - ext4
/dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
{noformat}
was:
Currently, if a container uses container image, we'll do a bind mount of its
sandbox (<sandbox> -> <rootfs>/mnt/mesos/sandbox) in the host mount namespace.
However, doing the mounts in the host mount table is not ideal. That
complicates both the cleanup path and the recovery path.
Instead, we can do the sandbox bind mount in the container's mount namespace so
that cleanup and recovery will be greatly simplified. We can setup mount
propagation properly so that persistent volumes mounted at <sandbox>/xxx can be
propagated into the container.
> Sandbox mounts should not be in the host mount namespace.
> ---------------------------------------------------------
>
> Key: MESOS-5307
> URL: https://issues.apache.org/jira/browse/MESOS-5307
> Project: Mesos
> Issue Type: Improvement
> Reporter: Jie Yu
> Assignee: Jie Yu
>
> Currently, if a container uses container image, we'll do a bind mount of its
> sandbox (<sandbox> -> <rootfs>/mnt/mesos/sandbox) in the host mount namespace.
> However, doing the mounts in the host mount table is not ideal. That
> complicates both the cleanup path and the recovery path.
> Instead, we can do the sandbox bind mount in the container's mount namespace
> so that cleanup and recovery will be greatly simplified. We can setup mount
> propagation properly so that persistent volumes mounted at <sandbox>/xxx can
> be propagated into the container.
> Here is a simple proof of concept:
> Console 1:
> {noformat}
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ ll .
> total 12
> drwxrwxr-x 3 vagrant vagrant 4096 Apr 25 16:05 ./
> drwxrwxr-x 6 vagrant vagrant 4096 Apr 25 23:17 ../
> drwxrwxr-x 5 vagrant vagrant 4096 Apr 25 23:17 slave/
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ ll slave/
> total 20
> drwxrwxr-x 5 vagrant vagrant 4096 Apr 25 23:17 ./
> drwxrwxr-x 3 vagrant vagrant 4096 Apr 25 16:05 ../
> drwxrwxr-x 6 vagrant vagrant 4096 Apr 26 21:06 directory/
> drwxr-xr-x 12 vagrant vagrant 4096 Apr 25 23:20 rootfs/
> drwxrwxr-x 2 vagrant vagrant 4096 Apr 25 16:09 volume/
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ sudo mount --bind slave/ slave/
>
>
>
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ sudo mount --make-shared slave/
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ cat /proc/self/mountinfo
> 50 22 8:1 /home/vagrant/tmp/mesos/slave /home/vagrant/tmp/mesos/slave
> rw,relatime shared:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> {noformat}
> Console 2:
> {noformat}
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ cd slave/
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave$ sudo unshare -m /bin/bash
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# sudo mount --make-rslave .
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# cat /proc/self/mountinfo
> 124 63 8:1 /home/vagrant/tmp/mesos/slave /home/vagrant/tmp/mesos/slave
> rw,relatime master:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# mount --rbind directory/
> rootfs/mnt/mesos/sandbox/
>
>
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# mount --rbind rootfs/ rootfs/
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# mount -t proc proc
> rootfs/proc
>
>
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# pivot_root rootfs
> rootfs/tmp/.rootfs
>
>
> root@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave# cd /
> root@vagrant-ubuntu-trusty-64:/# cat /proc/self/mountinfo
> 126 61 8:1 /home/vagrant/tmp/mesos/slave/rootfs / rw,relatime master:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> 127 126 8:1 /home/vagrant/tmp/mesos/slave/directory /mnt/mesos/sandbox
> rw,relatime master:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> 128 126 0:3 / /proc rw,relatime - proc proc rw
> {noformat}
> Console 1:
> {noformat}
> agrant@vagrant-ubuntu-trusty-64:~/tmp/mesos$ cd slave/
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave$ sudo mount --bind volume/
> directory/v1
> vagrant@vagrant-ubuntu-trusty-64:~/tmp/mesos/slave$ cat /proc/self/mountinfo
> 50 22 8:1 /home/vagrant/tmp/mesos/slave /home/vagrant/tmp/mesos/slave
> rw,relatime shared:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> 129 50 8:1 /home/vagrant/tmp/mesos/slave/volume
> /home/vagrant/tmp/mesos/slave/directory/v1 rw,relatime shared:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> {noformat}
> Console 2:
> {noformat}
> root@vagrant-ubuntu-trusty-64:/# cat /proc/self/mountinfo
> 126 61 8:1 /home/vagrant/tmp/mesos/slave/rootfs / rw,relatime master:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> 127 126 8:1 /home/vagrant/tmp/mesos/slave/directory /mnt/mesos/sandbox
> rw,relatime master:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> 128 126 0:3 / /proc rw,relatime - proc proc rw
> 132 127 8:1 /home/vagrant/tmp/mesos/slave/volume /mnt/mesos/sandbox/v1
> rw,relatime shared:4 master:1 - ext4
> /dev/disk/by-uuid/baf292e5-0bb6-4e58-8a71-5b912e0f09b6 rw,data=ordered
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
