> On Aug. 14, 2015, 11:15 a.m., Jie Yu wrote: > > src/slave/containerizer/provisioners/backend.hpp, lines 55-57 > > <https://reviews.apache.org/r/37382/diff/2/?file=1040433#file1040433line55> > > > > Hum, what do yo mean here? > > > > I think the backend should be container aware. I.e., we should pass in > > containerid in 'provision' and destroy will take a containerid as well. > > > > There could be multiple rootfs for a container (because image in > > volumes). When the container finishes, the provisioner will try to destroy > > all rootfs provisioned for that container. > > > > Thoughts? > > Jiang Yan Xu wrote: > This is intended for the image in volumes case but I was trying to not be > too explicit because it's not implemented yet (we should track it with a > separate ticket). > > My comment about 'nested' was incorrect, let me clarify here. > > The 'caller' is the provisioner. What I had in mind was: each Backend > call handles one image, if `Backend::provision()`s one image and then it > should `destroy()` the rootfs provisioned by one image. > > Suppose we have such a rootfs dir and each container has multiple > rootfses, some from ContainerInfo::volumnes and one from > ContainerInfo::mesos::image but they are stored side-by-side under > <container_id>/<image_type>. > > ``` > <--rootfs_dir> > |--<container_id> > |--<appc> > |--<image_id> > |--<image_id> > |--<docker> > ``` > > IMO since LinuxFilesystemIsolatorProcess::cleanup() calls each > `Provisioner::destory(containerId)` (appc|docker), the provisioner can look > at the subdir it manages and scan it and call `Backend::destroy(rootfsPath)` > for each image under it. > > So each `Backend::destroy(rootfsPath)` call destroys one rootfs and there > is no `nesting` under it. > Passing `containerId` into Backend isn't as clean as it should be the > provisioner who converts the id into a path and Backends are this dumb thing > which only knows paths. > > More clear? > > Jie Yu wrote: > So the directory layout listed above is managed by the provisioner or > backend? It cannot be backend because otherwise that means provisioner needs > to know about the layout used by the backend. > > So if it's managed by provisioner, the layout should probably be: > ``` > /var/lib/mesos/provisioners/ > |-- appc/ > |-- <container_id> > |-- <image1> > |-- <image2> > |-- docker/ > |-- <container_id> > |-- <image3> > |-- <image4> > ``` > > So we only allow one backend, or we allow multiple backends (e.g., what > if we want to use bind mount backend for appc but overlayfs backend for > docker, or we want to gradually roll out overlayfs backend so that two > backends co-exists for a while)? If the later, how can we figure out which > backend to use when destroying root filesystems? > > I just want to get the full picture sorted out. We can chat next week. > > Jiang Yan Xu wrote: > Note that the above rootfs I mentioned is the result of the Backend > either collapsing its layers or overlay them and in the provisioner's view > it's just one entity. > > So the backend know about layout within a rootfs (e.g., its knows that a > rootfs is "ImageA(a script) -> ImageB(essential software) ->ImageC(base > OS)"); the provisioner knows about / manages the direcotry layout of > 'rootfs'es but not inside 'rootfs'es (e.g. This container wants three > rootfses, two of which from volumes). > > Yeah I agree the interface design requires a full picture of > multi-image-per-contain design. I will jot down my thoughts in a doc and > let's chat next week. > > Timothy Chen wrote: > Hey we should talk about this as it affects all of our work, I think I > didn't realize we're ending up doing multiple provisioner images for a single > executor. > I think my assumption was that the backend when given a provision call is > for a image as Yan mentioned, and returning the rootfs directory path which > the provisioner can hold on to know where the backend has provisioned it to. > > We don't really allow multiple backends with our current design, I don't > think there is anything stopping us but it then become a very interesting API > choice to see where we allow the user to specify this. My intuition is to > just use one single backend and we record the backend that is used to > provisioner a rootfs, so at least on recovery we can say we can't recover due > to a backend change.
Jie and I chatted offline and I am going to submit another review regarding the file system layout that addresses multiple provisioners / mutliple backends. The current Backend interface design is compatible with it and not itself controversial so I am just going to make some documentation adjustments here. Regardign a few issues which I'll describe in details in another review: Multiple provisioners: To support [MESOS-3004](https://issues.apache.org/jira/browse/MESOS-3004) there are already multiple images within a container and there is no real obstacle to prevent them from being different image types. Multiple backends: I agree that we don't have a use case for allowing the users to choose the backend (so we don't need to design the API for it) but the case for updating the Backend for a running cluster is real. We don't have to turn away a rootfs known to one of the backends during recovery / cleanup just because it's not the currently selected backend for new containers. I'll explain in a new review. - Jiang Yan ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/37382/#review95455 ----------------------------------------------------------- On Aug. 14, 2015, 10:51 a.m., Jiang Yan Xu wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/37382/ > ----------------------------------------------------------- > > (Updated Aug. 14, 2015, 10:51 a.m.) > > > Review request for mesos, Lily Chen, Ian Downes, Jie Yu, and Timothy Chen. > > > Bugs: MESOS-2968 > https://issues.apache.org/jira/browse/MESOS-2968 > > > Repository: mesos > > > Description > ------- > > The Backend interface is made generic so it can be used by different > provisioners. See > [MESOS-2968](https://issues.apache.org/jira/browse/MESOS-2968?focusedCommentId=14652859&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14652859) > for details. > > > Diffs > ----- > > src/Makefile.am b5dbdc316cad179cd265bd81900999fab35940b9 > src/slave/containerizer/provisioners/backend.hpp PRE-CREATION > src/slave/containerizer/provisioners/backend.cpp PRE-CREATION > > Diff: https://reviews.apache.org/r/37382/diff/ > > > Testing > ------- > > make check. > > > Thanks, > > Jiang Yan Xu > >
