[
https://issues.apache.org/jira/browse/MESOS-6653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15749117#comment-15749117
]
Jie Yu commented on MESOS-6653:
-------------------------------
commit 812e5e3d4e4d9e044a1cfe6cc7eaab10efb499b6
Author: Gilbert Song <[email protected]>
Date: Wed Dec 14 10:49:19 2016 -0800
Fixed overlay backend symlink error message.
Review: https://reviews.apache.org/r/54211/
> Overlayfs backend may fail to mount the rootfs if both container image and
> image volume are specified.
> ------------------------------------------------------------------------------------------------------
>
> Key: MESOS-6653
> URL: https://issues.apache.org/jira/browse/MESOS-6653
> Project: Mesos
> Issue Type: Bug
> Components: containerization
> Reporter: Gilbert Song
> Assignee: Gilbert Song
> Labels: backend, containerizer, overlayfs
>
> Depending on MESOS-6000, we use symlink to shorten the overlayfs mounting
> arguments. However, if more than one image need to be provisioned (e.g., a
> container image is specified while image volumes are specified for the same
> container), the symlink .../backends/overlay/links would fail to be created
> since it exists already.
> Here is a simple log when we hard code overlayfs as our default backend:
> {noformat}
> [07:02:45] : [Step 10/10] [ RUN ]
> Nesting/VolumeImageIsolatorTest.ROOT_ImageInVolumeWithRootFilesystem/0
> [07:02:46] : [Step 10/10] I1127 07:02:46.416021 2919
> containerizer.cpp:207] Using isolation:
> filesystem/linux,volume/image,docker/runtime,network/cni
> [07:02:46] : [Step 10/10] I1127 07:02:46.419312 2919
> linux_launcher.cpp:150] Using /sys/fs/cgroup/freezer as the freezer hierarchy
> for the Linux launcher
> [07:02:46] : [Step 10/10] E1127 07:02:46.425336 2919 shell.hpp:107]
> Command 'hadoop version 2>&1' failed; this is the output:
> [07:02:46] : [Step 10/10] sh: 1: hadoop: not found
> [07:02:46] : [Step 10/10] I1127 07:02:46.425379 2919 fetcher.cpp:69]
> Skipping URI fetcher plugin 'hadoop' as it could not be created: Failed to
> create HDFS client: Failed to execute 'hadoop version 2>&1'; the command was
> either not found or exited with a non-zero exit status: 127
> [07:02:46] : [Step 10/10] I1127 07:02:46.425452 2919 local_puller.cpp:94]
> Creating local puller with docker registry '/tmp/R6OUei/registry'
> [07:02:46] : [Step 10/10] I1127 07:02:46.427258 2934
> containerizer.cpp:956] Starting container
> 9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330 for executor 'test_executor' of
> framework
> [07:02:46] : [Step 10/10] I1127 07:02:46.427592 2938
> metadata_manager.cpp:167] Looking for image 'test_image_rootfs'
> [07:02:46] : [Step 10/10] I1127 07:02:46.427774 2936 local_puller.cpp:147]
> Untarring image 'test_image_rootfs' from
> '/tmp/R6OUei/registry/test_image_rootfs.tar' to
> '/tmp/R6OUei/store/staging/9krDz2'
> [07:02:46] : [Step 10/10] I1127 07:02:46.512070 2933 local_puller.cpp:167]
> The repositories JSON file for image 'test_image_rootfs' is
> '{"test_image_rootfs":{"latest":"815b809d588c80fd6ddf4d6ac244ad1c01ae4cbe0f91cc7480e306671ee9c346"}}'
> [07:02:46] : [Step 10/10] I1127 07:02:46.512279 2933 local_puller.cpp:295]
> Extracting layer tar ball
> '/tmp/R6OUei/store/staging/9krDz2/815b809d588c80fd6ddf4d6ac244ad1c01ae4cbe0f91cc7480e306671ee9c346/layer.tar
> to rootfs
> '/tmp/R6OUei/store/staging/9krDz2/815b809d588c80fd6ddf4d6ac244ad1c01ae4cbe0f91cc7480e306671ee9c346/rootfs'
> [07:02:46] : [Step 10/10] I1127 07:02:46.617442 2937
> metadata_manager.cpp:155] Successfully cached image 'test_image_rootfs'
> [07:02:46] : [Step 10/10] I1127 07:02:46.617908 2938 provisioner.cpp:286]
> Image layers: 1
> [07:02:46] : [Step 10/10] I1127 07:02:46.617925 2938 provisioner.cpp:296]
> Should hit here
> [07:02:46] : [Step 10/10] I1127 07:02:46.617949 2938 provisioner.cpp:315]
> !!!!: bind
> [07:02:46] : [Step 10/10] I1127 07:02:46.617959 2938 provisioner.cpp:315]
> !!!!: overlay
> [07:02:46] : [Step 10/10] I1127 07:02:46.617967 2938 provisioner.cpp:315]
> !!!!: copy
> [07:02:46] : [Step 10/10] I1127 07:02:46.617974 2938 provisioner.cpp:318]
> Provisioning image rootfs
> '/mnt/teamcity/temp/buildTmp/Nesting_VolumeImageIsolatorTest_ROOT_ImageInVolumeWithRootFilesystem_0_1fMo0c/provisioner/containers/9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330/backends/overlay/rootfses/c71e83d2-5dbe-4eb7-a2fc-b8cc826771f7'
> for container 9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330 using overlay backend
> [07:02:46] : [Step 10/10] I1127 07:02:46.618408 2936 overlay.cpp:175]
> Created symlink
> '/mnt/teamcity/temp/buildTmp/Nesting_VolumeImageIsolatorTest_ROOT_ImageInVolumeWithRootFilesystem_0_1fMo0c/provisioner/containers/9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330/backends/overlay/links'
> -> '/tmp/DQ3blT'
> [07:02:46] : [Step 10/10] I1127 07:02:46.618472 2936 overlay.cpp:203]
> Provisioning image rootfs with overlayfs:
> 'lowerdir=/tmp/DQ3blT/0,upperdir=/mnt/teamcity/temp/buildTmp/Nesting_VolumeImageIsolatorTest_ROOT_ImageInVolumeWithRootFilesystem_0_1fMo0c/provisioner/containers/9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330/backends/overlay/scratch/c71e83d2-5dbe-4eb7-a2fc-b8cc826771f7/upperdir,workdir=/mnt/teamcity/temp/buildTmp/Nesting_VolumeImageIsolatorTest_ROOT_ImageInVolumeWithRootFilesystem_0_1fMo0c/provisioner/containers/9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330/backends/overlay/scratch/c71e83d2-5dbe-4eb7-a2fc-b8cc826771f7/workdir'
> [07:02:46] : [Step 10/10] I1127 07:02:46.619098 2933 linux.cpp:451]
> Ignored an image volume for container 9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330
> [07:02:46] : [Step 10/10] I1127 07:02:46.619745 2938
> metadata_manager.cpp:167] Looking for image 'test_image_volume'
> [07:02:46] : [Step 10/10] I1127 07:02:46.619925 2937 local_puller.cpp:147]
> Untarring image 'test_image_volume' from
> '/tmp/R6OUei/registry/test_image_volume.tar' to
> '/tmp/R6OUei/store/staging/2GNlJO'
> [07:02:46] : [Step 10/10] I1127 07:02:46.713526 2935 local_puller.cpp:167]
> The repositories JSON file for image 'test_image_volume' is
> '{"test_image_volume":{"latest":"815b809d588c80fd6ddf4d6ac244ad1c01ae4cbe0f91cc7480e306671ee9c346"}}'
> [07:02:46] : [Step 10/10] I1127 07:02:46.713726 2935 local_puller.cpp:295]
> Extracting layer tar ball
> '/tmp/R6OUei/store/staging/2GNlJO/815b809d588c80fd6ddf4d6ac244ad1c01ae4cbe0f91cc7480e306671ee9c346/layer.tar
> to rootfs
> '/tmp/R6OUei/store/staging/2GNlJO/815b809d588c80fd6ddf4d6ac244ad1c01ae4cbe0f91cc7480e306671ee9c346/rootfs'
> [07:02:46] : [Step 10/10] I1127 07:02:46.818696 2937
> metadata_manager.cpp:155] Successfully cached image 'test_image_volume'
> [07:02:46] : [Step 10/10] I1127 07:02:46.819169 2934 provisioner.cpp:286]
> Image layers: 1
> [07:02:46] : [Step 10/10] I1127 07:02:46.819188 2934 provisioner.cpp:296]
> Should hit here
> [07:02:46] : [Step 10/10] I1127 07:02:46.819221 2934 provisioner.cpp:315]
> !!!!: bind
> [07:02:46] : [Step 10/10] I1127 07:02:46.819232 2934 provisioner.cpp:315]
> !!!!: overlay
> [07:02:46] : [Step 10/10] I1127 07:02:46.819236 2934 provisioner.cpp:315]
> !!!!: copy
> [07:02:46] : [Step 10/10] I1127 07:02:46.819241 2934 provisioner.cpp:318]
> Provisioning image rootfs
> '/mnt/teamcity/temp/buildTmp/Nesting_VolumeImageIsolatorTest_ROOT_ImageInVolumeWithRootFilesystem_0_1fMo0c/provisioner/containers/9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330/backends/overlay/rootfses/baf632b3-29c5-45e4-9d2e-6f3a2bdd9759'
> for container 9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330 using overlay backend
> [07:02:46] : [Step 10/10]
> ../../src/tests/containerizer/volume_image_isolator_tests.cpp:214: Failure
> [07:02:46] : [Step 10/10] (launch).failure(): Failed to create symlink
> '/mnt/teamcity/temp/buildTmp/Nesting_VolumeImageIsolatorTest_ROOT_ImageInVolumeWithRootFilesystem_0_1fMo0c/provisioner/containers/9af6c98a-d9f7-4c89-a5ed-fc7ae2fa1330/backends/overlay/links'
> -> '/tmp/6dj9IG'
> [07:02:46] : [Step 10/10] [ FAILED ]
> Nesting/VolumeImageIsolatorTest.ROOT_ImageInVolumeWithRootFilesystem/0, where
> GetParam() = false (919 ms)
> {noformat}
> We should differenciate the links for different provisioned images.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
