> On Feb. 9, 2017, 7:36 a.m., Jie Yu wrote: > > src/slave/containerizer/mesos/provisioner/oci/paths.hpp, lines 36-39 > > <https://reviews.apache.org/r/54639/diff/3/?file=1621023#file1621023line36> > > > > Can you elaborate a bit more the layout here? > > > > so `layer_id` is the digest? What is `config.json`? Does it store > > decompressed content, or the original content? what about image configs? > > > > Given this > > https://github.com/opencontainers/image-spec/blob/master/image-layout.md, > > my hunch is that we should probably cache the original blob, as well as > > decompressed content, potentially in separate subdirectories. > > > > ``` > > <oci_store_dir> > > |--- blobs/ # this is origional CAS blobs. > > |--- sha256/ > > |--- <digest> > > |--- <digest> > > |--- ... > > |--- layers/ # this is decompressed filesystem change sets > > |--- <id? digest?> > > |--- ... > > ``` > > > > Looks like we probably do not need `configs` or `manifests` directory > > for now because the only media type is json, and can be directly read from > > `blobs/`. > > > > Another thing to consider is the conversion to overlayfs opaque file > > from whiteout files as you did before. So for `layers` directory, we > > probably need to have a `backend` in the path (e.g., > > `layers/overlayfs/...`). > > Jie Yu wrote: > Also, worth taking a look at how Docker (or other OCI system) organize > its cache. > > Qian Zhang wrote: > Please take a look at https://reviews.apache.org/r/56147/diff/1#4, I have > changed it to: > ``` > |--layers > |--<layer_id> > |-- rootfs > |--<layer_id> > |-- rootfs > |--<config.json> > ``` > Here both `<layer_id>` and `<config.json>` are digest. > `<layer_id>/rootfs/` stores decompressed content(filesystem) extracted by > `command::untar()` from the layer tar ball. `<config.json>` stores the image > configuration. > > Can you please let me know why we need to cache the original blobs? In my > current implementation, the orignal blobs will only be fetched by fetcher > into a temp directory under `staging` directory, `layers` directory will only > store the decompressed content, and that temp directory under `staging` > directory will be removed after fetching image is done. I do not think we > need the original blobs once the fetching image is done and we have cached > the image in the store. > > And actually under `layers/<layer_id>`, there can be two sub-directories: > `rootfs` (for aufs/bind/copy) and `rootfs.overlay` (for overlay), this is > same with what I did for for Docker store. Do you think we need to change it > to `layers/non-overlay/<layer_id>` and `layers/overlay/<layer_id>` for OCI > store?
And I checked Docker today, it does not cache blobs as well. When pulling an image, Docker will download the blobs in a temp directory (`/var/lib/docker/tmp/`), and once the pulling is done, the blobs in that temp directory will be removed. And Docker will store the decompressed content of each image layer in `/var/lib/docker/aufs/diff/<layerID>`. - Qian ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/54639/#review164794 ----------------------------------------------------------- On Feb. 1, 2017, 9:17 a.m., Qian Zhang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/54639/ > ----------------------------------------------------------- > > (Updated Feb. 1, 2017, 9:17 a.m.) > > > Review request for mesos, Gilbert Song and Jie Yu. > > > Bugs: MESOS-6681 > https://issues.apache.org/jira/browse/MESOS-6681 > > > Repository: mesos > > > Description > ------- > > Implemented the 'create()' method of OCI store. > > > Diffs > ----- > > src/CMakeLists.txt 09ef1aee680c6b91476092dbf61d4ca3a8d256bb > src/Makefile.am 6c9be54fe0ce3faaa26e2f090773d1b77bb6e430 > src/slave/containerizer/mesos/provisioner/oci/paths.hpp PRE-CREATION > src/slave/containerizer/mesos/provisioner/oci/paths.cpp PRE-CREATION > src/slave/containerizer/mesos/provisioner/oci/store.cpp PRE-CREATION > > Diff: https://reviews.apache.org/r/54639/diff/ > > > Testing > ------- > > > Thanks, > > Qian Zhang > >
