Since you are using docker image which means that your container will have rootfs, so it is not required to have the absolute path exist, the linux file system isolator will help create the path automatically https://github.com/apache/mesos/blob/0.28.x/src/slave/containerizer/mesos/isolators/filesystem/linux.cpp#L390-L402
Can you please share your framework? How did you set the volume part in your framework? Thanks, Guangya On Fri, May 20, 2016 at 4:54 AM, Olivier Sallou <[email protected]> wrote: > > > ----- Mail original ----- > > De: "Gilbert Song" <[email protected]> > > À: "dev" <[email protected]> > > Envoyé: Jeudi 19 Mai 2016 01:57:16 > > Objet: Re: volume / mount point error with Unified Containerizer > > > > @Olivier, > > In mesos 0.28.1, you are supposed to be able bind mount a volume from > > the host into the mesos container. Did you specify a docker image (we > > determine > > the mount point differently depending whether the container has a > rootfs)? > > Yes I specified an image, a Docker image URI. > > > How > > do you specify your 'container_path' (the mount point in the container)? > If > > it is an > > absolute path, we require that dir to be pre-existed. If it is a relative > > path, we will > > mkdir for it. > > It is an absolute path, but it does not exists in image (this is the > issue). Images are custom Docker images (images containing tools for batch > computing), and I want, for example, to mount some shared resources (user > home dir, common data, etc.) in the image. Of course those directories do > not pre-exists in container images as they are specific to the environment. > Requiring existence of the directory in the image is not issue as it > prevents using any existing image from a repo. > > When using Docker containerizer it works fine, I can mount any external > storage in the container. > > Olivie > > > > > > @Joshua, > > Thank for posting your workaround on mesos. As I mentioned above, in > 0.28.1 > > or > > older, we only mkdir for container_path which is relative path (not > > starting with "/"). > > Because if no rootfs specified for a mesos container, the container > shares > > the host > > root filesystem. Obviously we don't want any random files to be created > > implicitly > > on your host fs. > > From mesos 0.29 (release by the end of this month), we will mkdir the > mount > > point in the container except for the command task case that specify an > > absolute > > container_path without a rootfs. Because we simplify the mounting logic, > and > > sandbox bind mount will only be done in container mount namespace > instead of > > host mount namespace (what we did before). Please keep tuned. > > > > Cheers, > > Gilbert > > > > On Wed, May 18, 2016 at 8:14 AM, Joshua Cohen <[email protected]> wrote: > > > > > Hi Olivier, > > > > > > I touched on this issue as part of > > > https://issues.apache.org/jira/browse/MESOS-5229. It would be nice if > > > Mesos > > > automatically created container mount points if they don't already > exist. > > > In the meantime, as a workaround for this, I've updated my filesystem > > > images to include the path (e.g. in Dockerfile, add `RUN mkdir -p > > > /some/mount/point`). Not the best solution, but the only thing I've > seen > > > that works at the moment. > > > > > > Cheers, > > > > > > Joshua > > > > > > On Wed, May 18, 2016 at 7:36 AM, Guangya Liu <[email protected]> > wrote: > > > > > > > It's pretty simple for you from scratch with source code > > > > > > > > > > > > https://github.com/apache/mesos/blob/master/docs/getting-started.md#building-mesos > > > > ;-) > > > > > > > > Thanks, > > > > > > > > Guangya > > > > > > > > On Wed, May 18, 2016 at 8:30 PM, Olivier Sallou < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > On 05/18/2016 02:31 PM, Guangya Liu wrote: > > > > > > Just saw that you are working with 0.28.1, the "docker volume > driver" > > > > > code > > > > > > was not in 0.28.1, can you please have a try with mesos master > branch > > > > if > > > > > > you are only doing some test? > > > > > this is indeed test only for the moment. But I will have to > > > > > recompile/install mesos :-( (I used packages for install). > > > > > > > > > > I will try when possible, but thanks for the hint. > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Guangya > > > > > > > > > > > > On Wed, May 18, 2016 at 8:28 PM, Guangya Liu <[email protected] > > > > > > wrote: > > > > > > > > > > > >> Hi Olivier, > > > > > >> > > > > > >> I think that you need to enable "docker volume isolator" if you > want > > > > use > > > > > >> external storage with unified container I was writing a document > > > here > > > > > >> https://reviews.apache.org/r/47511/, perhaps you can have a try > > > > > according > > > > > >> to the document and post some comments there if you find any > issues. > > > > > >> > > > > > >> Also you can patch mesos-execute here > > > > > https://reviews.apache.org/r/46762/ to > > > > > >> have a try with mesos-execute. > > > > > >> > > > > > >> Thanks, > > > > > >> > > > > > >> Guangya > > > > > >> > > > > > >> On Wed, May 18, 2016 at 7:17 PM, Olivier Sallou < > > > > > [email protected]> > > > > > >> wrote: > > > > > >> > > > > > >>> Answering (partially) to myself. > > > > > >>> > > > > > >>> I seems issue is container_path does not exists inside > container. > > > On > > > > > >>> Docker, path is created and mounted. With pure mesos, > > > container_path > > > > > >>> must exists. > > > > > >>> > > > > > >>> mesos.proto says: "If the path is an absolute path, that path > must > > > > > >>> already exist." > > > > > >>> > > > > > >>> This is an issue however, using Docker images, the path I want > to > > > > mount > > > > > >>> does not exists, and it cannot be modified "on the fly". > > > > > >>> > > > > > >>> Is there a workaround for this ? > > > > > >>> > > > > > >>> > > > > > >>> On 05/18/2016 12:24 PM, Olivier Sallou wrote: > > > > > >>>> Hi, > > > > > >>>> I am trying unified containerizer on a single server > > > (master/slave) > > > > on > > > > > >>>> mesos 0.28.1, to switch from docker containerizer to > mesos+docker > > > > > image > > > > > >>>> container. > > > > > >>>> > > > > > >>>> I have setup slave config as suggested in documentation: > > > > > >>>> > > > > > >>>> containerizers=docker,mesos > > > > > >>>> image_providers=docker \ > > > > > >>>> isolation=filesystem/linux,docker/runtime > > > > > >>>> > > > > > >>>> However, when I execute my task with a volume I have an error: > > > > > >>>> > > > > > >>>> .... > > > > > >>>> + mount -n --rbind > > > > > >>>> > > > > > >>> > > > > > > > > > > > > > /tmp/mesos/provisioner/containers/2d7ea311-5e8b-440f-a3ca-a40e1b946b8e/backends/copy/rootfses/f9f66bb2-308d-4555-ba77-49ec61cbeb4f > > > > > >>> > > > > > > > > > > > > > /tmp/mesos/slaves/2a296daf-7419-4659-ade1-763c792cd522-S0/frameworks/aef1b0e3-ea2d-4770-baac-96d673ab88f9-0000/executors/51/runs/2d7ea311-5e8b-440f-a3ca-a40e1b946b8e/.rootfs > > > > > >>>> + mount -n --rbind > > > > > >>>> > > > > > >>> > > > > > > > > > > > > > /home/osallou/Development/NOSAVE/go-docker/godshared/tasks/pairtree_root/us/er/_o/sa/ll/ou/task > > > > > >>> > > > > > > > > > > > > > /tmp/mesos/slaves/2a296daf-7419-4659-ade1-763c792cd522-S0/frameworks/aef1b0e3-ea2d-4770-baac-96d673ab88f9-0000/executors/51/runs/2d7ea311-5e8b-440f-a3ca-a40e1b946b8e/.rootfs/mnt/god-data > > > > > >>>> mount: mount point > > > > > >>>> > > > > > >>> > > > > > > > > > > > > > /tmp/mesos/slaves/2a296daf-7419-4659-ade1-763c792cd522-S0/frameworks/aef1b0e3-ea2d-4770-baac-96d673ab88f9-0000/executors/51/runs/2d7ea311-5e8b-440f-a3ca-a40e1b946b8e/.rootfs/mnt/god-data > > > > > >>>> does not exist > > > > > >>>> Failed to execute a preparation shell command > > > > > >>>> > > > > > >>>> Then, my task switches to FAILED. > > > > > >>>> > > > > > >>>> I define a local volume to bind mount in my "container" > > > > > >>>> > > > > > >>> > > > > > > > > > > > > > /home/osallou/Development/NOSAVE/go-docker/godshared/tasks/pairtree_root/us/er/_o/sa/ll/ou/task > > > > > >>>> => /mnt/god-data > > > > > >>>> My directory exists on local server. > > > > > >>>> In mesos UI, I can see the .rootfs directory along stdout and > > > stderr > > > > > >>>> files, and inside .rootfs, I can see /mnt/god-data (empty). > > > > > >>>> > > > > > >>>> Running the same using Docker containerizer instead of mesos > > > > > >>>> containerizer (with a Docker image) works fine. > > > > > >>>> > > > > > >>>> It seems it fails to mount my local directory in the > container. > > > Any > > > > > idea > > > > > >>>> of what is going wrong or how to debug this? > > > > > >>>> > > > > > >>>> > > > > > >>>> Thanks > > > > > >>>> > > > > > >>> -- > > > > > >>> Olivier Sallou > > > > > >>> IRISA / University of Rennes 1 > > > > > >>> Campus de Beaulieu, 35000 RENNES - FRANCE > > > > > >>> Tel: 02.99.84.71.95 > > > > > >>> > > > > > >>> gpg key id: 4096R/326D8438 (keyring.debian.org) > > > > > >>> Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D > > > 8438 > > > > > >>> > > > > > >>> > > > > > > > > > > -- > > > > > Olivier Sallou > > > > > IRISA / University of Rennes 1 > > > > > Campus de Beaulieu, 35000 RENNES - FRANCE > > > > > Tel: 02.99.84.71.95 > > > > > > > > > > gpg key id: 4096R/326D8438 (keyring.debian.org) > > > > > Key fingerprint = 5FB4 6F83 D3B9 5204 6335 D26D 78DC 68DB 326D > 8438 > > > > > > > > > > > > > > > > > > > >
