[jira] [Updated] (YARN-8784) DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk
[ https://issues.apache.org/jira/browse/YARN-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated YARN-8784: -- Attachment: YARN-8784.001.patch > DockerLinuxContainerRuntime prevents access to distributed cache entries on a > full disk > --- > > Key: YARN-8784 > URL: https://issues.apache.org/jira/browse/YARN-8784 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.0, 3.1.1 >Reporter: Jason Lowe >Assignee: Eric Badger >Priority: Major > Labels: Docker > Attachments: YARN-8784.001.patch > > > DockerLinuxContainerRuntime bind mounts the filecache and usercache > directories into the container to allow tasks to access entries in the > distributed cache. However it only bind mounts directories on disks that > are considered good, and disks that are full or bad are not in that list. If > a container tries to run with a distributed cache entry that has been > previously localized to a disk that is now considered full/bad, the dist > cache directory will _not_ be bind-mounted into the container's filesystem > namespace. At that point any symlinks in the container's current working > directory that point to those disks will reference invalid paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8784) DockerLinuxContainerRuntime prevents access to distributed cache entries on a full disk
[ https://issues.apache.org/jira/browse/YARN-8784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated YARN-8784: -- Labels: Docker (was: ) > DockerLinuxContainerRuntime prevents access to distributed cache entries on a > full disk > --- > > Key: YARN-8784 > URL: https://issues.apache.org/jira/browse/YARN-8784 > Project: Hadoop YARN > Issue Type: Bug > Components: nodemanager >Affects Versions: 3.2.0, 3.1.1 >Reporter: Jason Lowe >Priority: Major > Labels: Docker > > DockerLinuxContainerRuntime bind mounts the filecache and usercache > directories into the container to allow tasks to access entries in the > distributed cache. However it only bind mounts directories on disks that > are considered good, and disks that are full or bad are not in that list. If > a container tries to run with a distributed cache entry that has been > previously localized to a disk that is now considered full/bad, the dist > cache directory will _not_ be bind-mounted into the container's filesystem > namespace. At that point any symlinks in the container's current working > directory that point to those disks will reference invalid paths. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
