> On Aug. 14, 2015, 11:15 a.m., Jie Yu wrote:
> > src/slave/containerizer/provisioners/backend.hpp, lines 55-57
> > <https://reviews.apache.org/r/37382/diff/2/?file=1040433#file1040433line55>
> >
> >     Hum, what do yo mean here?
> >     
> >     I think the backend should be container aware. I.e., we should pass in 
> > containerid in 'provision'  and destroy will take a containerid as well.
> >     
> >     There could be multiple rootfs for a container (because image in 
> > volumes). When the container finishes, the provisioner will try to destroy 
> > all rootfs provisioned for that container.
> >     
> >     Thoughts?
> 
> Jiang Yan Xu wrote:
>     This is intended for the image in volumes case but I was trying to not be 
> too explicit because it's not implemented yet (we should track it with a 
> separate ticket).
>     
>     My comment about 'nested' was incorrect, let me clarify here.
>     
>     The 'caller' is the provisioner. What I had in mind was: each Backend 
> call handles one image, if `Backend::provision()`s one image and then it 
> should `destroy()` the rootfs provisioned by one image.
>     
>     Suppose we have such a rootfs dir and each container has multiple 
> rootfses, some from ContainerInfo::volumnes and one from 
> ContainerInfo::mesos::image but they are stored side-by-side under 
> <container_id>/<image_type>.
>     
>     ```
>     <--rootfs_dir>
>     |--<container_id>
>        |--<appc>
>           |--<image_id>
>           |--<image_id>
>        |--<docker>
>     ```
>     
>     IMO since LinuxFilesystemIsolatorProcess::cleanup() calls each 
> `Provisioner::destory(containerId)` (appc|docker), the provisioner can look 
> at the subdir it manages and scan it and call `Backend::destroy(rootfsPath)` 
> for each image under it.
>     
>     So each `Backend::destroy(rootfsPath)` call destroys one rootfs and there 
> is no `nesting` under it.
>     Passing `containerId` into Backend isn't as clean as it should be the 
> provisioner who converts the id into a path and Backends are this dumb thing 
> which only knows paths.
>     
>     More clear?
> 
> Jie Yu wrote:
>     So the directory layout listed above is managed by the provisioner or 
> backend? It cannot be backend because otherwise that means provisioner needs 
> to know about the layout used by the backend.
>     
>     So if it's managed by provisioner, the layout should probably be:
>     ```
>     /var/lib/mesos/provisioners/
>         |-- appc/
>               |-- <container_id>
>                      |-- <image1>
>                      |-- <image2>
>         |-- docker/
>               |-- <container_id>
>                      |-- <image3>
>                      |-- <image4>
>     ```
>     
>     So we only allow one backend, or we allow multiple backends (e.g., what 
> if we want to use bind mount backend for appc but overlayfs backend for 
> docker, or we want to gradually roll out overlayfs backend so that two 
> backends co-exists for a while)? If the later, how can we figure out which 
> backend to use when destroying root filesystems?
>     
>     I just want to get the full picture sorted out. We can chat next week.
> 
> Jiang Yan Xu wrote:
>     Note that the above rootfs I mentioned is the result of the Backend 
> either collapsing its layers or overlay them and in the provisioner's view 
> it's just one entity.
>     
>     So the backend know about layout within a rootfs (e.g., its knows that a 
> rootfs is "ImageA(a script) -> ImageB(essential software) ->ImageC(base 
> OS)"); the provisioner knows about / manages the direcotry layout of 
> 'rootfs'es but not inside 'rootfs'es (e.g. This container wants three 
> rootfses, two of which from volumes).
>     
>     Yeah I agree the interface design requires a full picture of 
> multi-image-per-contain design. I will jot down my thoughts in a doc and 
> let's chat next week.
> 
> Timothy Chen wrote:
>     Hey we should talk about this as it affects all of our work, I think I 
> didn't realize we're ending up doing multiple provisioner images for a single 
> executor.
>     I think my assumption was that the backend when given a provision call is 
> for a image as Yan mentioned, and returning the rootfs directory path which 
> the provisioner can hold on to know where the backend has provisioned it to.
>     
>     We don't really allow multiple backends with our current design, I don't 
> think there is anything stopping us but it then become a very interesting API 
> choice to see where we allow the user to specify this. My intuition is to 
> just use one single backend and we record the backend that is used to 
> provisioner a rootfs, so at least on recovery we can say we can't recover due 
> to a backend change.

Jie and I chatted offline and I am going to submit another review regarding the 
file system layout that addresses multiple provisioners / mutliple backends. 
The current Backend interface design is compatible with it and not itself 
controversial so I am just going to make some documentation adjustments here.

Regardign a few issues which I'll describe in details in another review:

Multiple provisioners: To support 
[MESOS-3004](https://issues.apache.org/jira/browse/MESOS-3004) there are 
already multiple images within a container and there is no real obstacle to 
prevent them from being different image types.

Multiple backends: I agree that we don't have a use case for allowing the users 
to choose the backend (so we don't need to design the API for it) but the case 
for updating the Backend for a running cluster is real. We don't have to turn 
away a rootfs known to one of the backends during recovery / cleanup just 
because it's not the currently selected backend for new containers. I'll 
explain in a new review.


- Jiang Yan


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37382/#review95455
-----------------------------------------------------------


On Aug. 14, 2015, 10:51 a.m., Jiang Yan Xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/37382/
> -----------------------------------------------------------
> 
> (Updated Aug. 14, 2015, 10:51 a.m.)
> 
> 
> Review request for mesos, Lily Chen, Ian Downes, Jie Yu, and Timothy Chen.
> 
> 
> Bugs: MESOS-2968
>     https://issues.apache.org/jira/browse/MESOS-2968
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> The Backend interface is made generic so it can be used by different 
> provisioners. See 
> [MESOS-2968](https://issues.apache.org/jira/browse/MESOS-2968?focusedCommentId=14652859&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14652859)
>  for details.
> 
> 
> Diffs
> -----
> 
>   src/Makefile.am b5dbdc316cad179cd265bd81900999fab35940b9 
>   src/slave/containerizer/provisioners/backend.hpp PRE-CREATION 
>   src/slave/containerizer/provisioners/backend.cpp PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/37382/diff/
> 
> 
> Testing
> -------
> 
> make check.
> 
> 
> Thanks,
> 
> Jiang Yan Xu
> 
>

Reply via email to