Qian Zhang commented on MESOS-6360:

Here is a specific issue that I found for the current implementation of 
whiteout handling. Suppose I build a Docker image from the following Dockerfile:
FROM cirros
RUN touch /opt/data
RUN rm -rf /opt/data
RUN echo yes > /opt/data

And I launch a container from that image via Mesos using overlay provisioner 
backend, then I found there is NO {{data}} file under {{/opt}} in the 
container, but if I launch a Docker container from that image via Docker 
engine, then I found there is the {{data}} file under {{/opt}} in the container 
and its content is {{yes}}. Obviously the result of Docker container is correct.

The root cause of this issue is the current way that we handle the whiteout 
file generated by the line 3 in the above Dockerfile is not correct: we always 
handle the whiteout files after the whole rootfs for the container has been 
provisioned, but that will lead to wrongly deleting a file which should not be 

> The handling of whiteout files in provisioner is not correct
> ------------------------------------------------------------
>                 Key: MESOS-6360
>                 URL: https://issues.apache.org/jira/browse/MESOS-6360
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Qian Zhang
>            Assignee: Qian Zhang
>            Priority: Blocker
> Currently when user launches a container from a Docker image via universal 
> containerizer, we always handle the whiteout files in 
> {{ProvisionerProcess::__provision()}} regardless of which backend is used.
> However this is actually not correct, because the way to handle whiteout 
> files is backend dependent, that means for different backends, we need to 
> handle whiteout files in different ways, e.g.:
> * AUFS backend: It seems the AUFS whiteout ({{.wh.<filename/dirname>}} and 
> {{.wh..wh..opq}}) is the whiteout standard in Docker (see [this comment | 
> https://github.com/docker/docker/blob/v1.12.1/pkg/archive/archive.go#L259:L262]
>  for details), so that means after the Docker image is pulled, its whiteout 
> files in the store are already in aufs format, then we do not need to do 
> anything about whiteout file handling because the aufs mount done in 
> {{AufsBackendProcess::provision()}} will handle it automatically.
> * Overlay backend: Overlayfs has its own whiteout files (see [this doc | 
> https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt] for 
> details), so we need to convert the aufs whiteout files to overlayfs whiteout 
> files before we do the overlay mount in {{OverlayBackendProcess::provision}} 
> which will automatically handle the overlayfs whiteout files.
> * Copy backend: We need to manually handle the aufs whiteout files when we 
> copy each layer in {{CopyBackendProcess::_provision()}}.

This message was sent by Atlassian JIRA

Reply via email to