[
https://issues.apache.org/jira/browse/YARN-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16973463#comment-16973463
]
Shane Kumpf edited comment on YARN-9562 at 11/13/19 3:55 PM:
-------------------------------------------------------------
Hey Eric, sorry for the delay. Just a note that my patch testing workflow is to
always start with a fresh checkout of the source, apply the patches, and start
a brand new VM, so there wasn't a switch from local to non-local.
Here is a listing of what remains after I get that exception during cleanup.
{code}
[root@y7001 ~]# find /tmp/hadoop-yarn/nm-local-dir/ -ls
67826613 0 drwxr-xr-x 5 yarn yarn 57 Nov 13 15:36
/tmp/hadoop-yarn/nm-local-dir/
8703405 0 drwxr-xr-x 3 yarn yarn 24 Nov 13 15:47
/tmp/hadoop-yarn/nm-local-dir/usercache
108869938 0 drwxr-s--- 4 nobody hadoop 39 Nov 13 15:47
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser
70809061 0 drwxr-s--- 3 nobody hadoop 44 Nov 13 15:49
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache
8703450 0 drwxr-s--- 3 nobody hadoop 52 Nov 13 15:50
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003
38635536 0 drwx--s--- 3 nobody hadoop 31 Nov 13 15:50
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002
8711268 0 drwx--x--- 3 nobody hadoop 25 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp
108869950 0 drwxr-xr-x 3 root root 26 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn
8711270 0 drwxr-xr-x 4 root root 40 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir
38635540 0 drwxr-xr-x 3 root root 24 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache
70809098 0 drwxr-xr-x 4 root root 39 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser
108869951 0 drwxr-xr-x 3 root root 44 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache
8711271 0 drwxr-xr-x 2 root root 6 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003
70809099 0 drwxr-xr-x 2 root root 6 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/filecache
38635541 0 drwxr-xr-x 2 root root 6 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/filecache
33868121 0 drwx--x--- 2 nobody hadoop 6 Nov 13 15:47
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/filecache
38388470 0 drwxr-xr-x 4 yarn yarn 26 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache
108869918 0 drwxr-xr-x 2 yarn yarn 155 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache/11
108869919 4 -r-xr-xr-x 1 yarn yarn 2183 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache/11/5e35e350aded98340bc8fcb0ba392d809c807bc3eb5c618d4a0674d98d88bccd
108869944 4 -rw-r--r-- 1 yarn yarn 28 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache/11/.5e35e350aded98340bc8fcb0ba392d809c807bc3eb5c618d4a0674d98d88bccd.crc
70809093 0 drwxr-xr-x 2 yarn yarn 165 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache/10
70809094 72260 -r-xr-xr-x 1 yarn yarn 73994240 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache/10/ab5ef0e5819490abe86106fd9f4381123e37a03e80e650be39f7938d30ecb530.sqsh
70809095 568 -rw-r--r-- 1 yarn yarn 578088 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/filecache/10/.ab5ef0e5819490abe86106fd9f4381123e37a03e80e650be39f7938d30ecb530.sqsh.crc
67826614 0 drwx------ 2 yarn yarn 6 Nov 13 15:50
/tmp/hadoop-yarn/nm-local-dir/nmPrivate
{code}
I haven't tested removing it and am out of time at the moment, but it seems
related to the private_slash_tmp dir, as that is what remains.
was (Author: [email protected]):
Hey Eric, sorry for the delay. Just a note that my patch testing workflow is to
always start with a fresh checkout of the source, apply the patches, and start
a brand new VM, so there wasn't a switch from local to non-local.
Here is a listing of what remains after I get that exception during cleanup.
{code}
[root@y7001 ~]# find
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/ -ls
70809061 0 drwxr-s--- 3 nobody hadoop 44 Nov 13 15:49
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/
8703450 0 drwxr-s--- 3 nobody hadoop 52 Nov 13 15:50
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003
38635536 0 drwx--s--- 3 nobody hadoop 31 Nov 13 15:50
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002
8711268 0 drwx--x--- 3 nobody hadoop 25 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp
108869950 0 drwxr-xr-x 3 root root 26 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn
8711270 0 drwxr-xr-x 4 root root 40 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir
38635540 0 drwxr-xr-x 3 root root 24 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache
70809098 0 drwxr-xr-x 4 root root 39 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser
108869951 0 drwxr-xr-x 3 root root 44 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache
8711271 0 drwxr-xr-x 2 root root 6 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003
70809099 0 drwxr-xr-x 2 root root 6 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/filecache
38635541 0 drwxr-xr-x 2 root root 6 Nov 13 15:48
/tmp/hadoop-yarn/nm-local-dir/usercache/hadoopuser/appcache/application_1573659389683_0003/container_1573659389683_0003_01_000002/private_slash_tmp/hadoop-yarn/nm-local-dir/filecache
{code}
I haven't tested removing it and am out of time at the moment, but it seems
related to the private_slash_tmp dir, as that is what remains.
> Add Java changes for the new RuncContainerRuntime
> -------------------------------------------------
>
> Key: YARN-9562
> URL: https://issues.apache.org/jira/browse/YARN-9562
> Project: Hadoop YARN
> Issue Type: Sub-task
> Reporter: Eric Badger
> Assignee: Eric Badger
> Priority: Major
> Attachments: YARN-9562.001.patch, YARN-9562.002.patch,
> YARN-9562.003.patch, YARN-9562.004.patch, YARN-9562.005.patch,
> YARN-9562.006.patch, YARN-9562.007.patch, YARN-9562.008.patch,
> YARN-9562.009.patch, YARN-9562.010.patch, YARN-9562.011.patch,
> YARN-9562.012.patch, YARN-9562.013.patch, YARN-9562.014.patch
>
>
> This JIRA will be used to add the Java changes for the new
> RuncContainerRuntime. This will work off of YARN-9560 to use much of the
> existing DockerLinuxContainerRuntime code once it is moved up into an
> abstract class that can be extended.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
