Zhitao Li updated MESOS-4945:
    Comment: was deleted

(was: Revised plan in rough steps:
* For each image, checkpoint a) container ids, b) time of last container using 
it being destroyed, and c) size of each layer;
    ** TODO: how do deal with migration? idea is passing in more info in 
recover() chain of containerizer -> provisioner -> store;
* Change store interface:
    ** "get(Image)" to "get(Image, ContainerID)": The containerID field added 
can be used to implement ref counting and further book keeping (i.e. get local 
images information);
    ** add "remove(Image, ContainerID)" virtual function: this is optional in 
that store which does not do ref counting can have an empty implementation.
*  Make sure provisioner::destroy() call store::remove(Image, ContainerID);
* Add command line flag for docker store capacity limit (in bytes);
* In (docker) store::get(Image, ContainerID), after a pull is done, calculate 
total layer sizes, if above store capacity, remove unused images (determined by 
empty container ids), sorted by last time not used. Any layer not shared by 
leftover images is also removed, until total size is dropped below capacity.

Open question: 

1) In this design, we have one explicit reference counting between 
{{Container}} and {{Image}} in store. However, this information could be 
constructed on-the-fly with all containers in {{Containerizer}} class. Do we 
consider this "double accounting" problematic, or error-prone?
2) Is calling new {{remove(Image, ContainerID)}} from 
{{Provisioner::destroy()}} sufficient to make sure all book keepings are 
properly done?)

> Garbage collect unused docker layers in the store.
> --------------------------------------------------
>                 Key: MESOS-4945
>                 URL: https://issues.apache.org/jira/browse/MESOS-4945
>             Project: Mesos
>          Issue Type: Epic
>            Reporter: Jie Yu
>            Assignee: Zhitao Li
> Right now, we don't have any garbage collection in place for docker layers. 
> It's not straightforward to implement because we don't know what container is 
> currently using the layer. We probably need a way to track the current usage 
> of layers.

This message was sent by Atlassian JIRA

Reply via email to