[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191708#comment-14191708 ] Hudson commented on YARN-661: - FAILURE: Integrated in Hadoop-Yarn-trunk #729 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/729/]) YARN-2755. NM fails to clean up usercache_DEL_timestamp dirs after YARN-661. Contributed by Siqi Li (jlowe: rev 73e626ad91cd5c06a005068d8432fd16e06fe6a0) * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.0-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191783#comment-14191783 ] Hudson commented on YARN-661: - FAILURE: Integrated in Hadoop-Hdfs-trunk #1918 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1918/]) YARN-2755. NM fails to clean up usercache_DEL_timestamp dirs after YARN-661. Contributed by Siqi Li (jlowe: rev 73e626ad91cd5c06a005068d8432fd16e06fe6a0) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.0-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14191914#comment-14191914 ] Hudson commented on YARN-661: - FAILURE: Integrated in Hadoop-Mapreduce-trunk #1943 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1943/]) YARN-2755. NM fails to clean up usercache_DEL_timestamp dirs after YARN-661. Contributed by Siqi Li (jlowe: rev 73e626ad91cd5c06a005068d8432fd16e06fe6a0) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.0-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14190220#comment-14190220 ] Hudson commented on YARN-661: - FAILURE: Integrated in Hadoop-trunk-Commit #6393 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6393/]) YARN-2755. NM fails to clean up usercache_DEL_timestamp dirs after YARN-661. Contributed by Siqi Li (jlowe: rev 73e626ad91cd5c06a005068d8432fd16e06fe6a0) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * hadoop-yarn-project/CHANGES.txt NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.0-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710957#comment-13710957 ] Hudson commented on YARN-661: - SUCCESS: Integrated in Hadoop-Yarn-trunk #273 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/273/]) YARN-661. Fixed NM to cleanup users' local directories correctly when starting up. Contributed by Omkar Vinit Joshi. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1503942) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDeletionService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.1-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13711064#comment-13711064 ] Hudson commented on YARN-661: - FAILURE: Integrated in Hadoop-Hdfs-trunk #1463 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1463/]) YARN-661. Fixed NM to cleanup users' local directories correctly when starting up. Contributed by Omkar Vinit Joshi. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1503942) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDeletionService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.1-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710323#comment-13710323 ] Omkar Vinit Joshi commented on YARN-661: bq. DeletionService.fileDeletionTaskFinished(): before successorTask gets scheduled, you are checking for successorTask.getSucess(). Assuming current task finished successfully, we don't set the success status on successorTask(lines +288 - +290). And the default for success is false. So successorTask won't even be scheduled when current task finishes ?! But the test passes, confused. If the code is correct, comments will help. No the default is success=true. If say we have 3 dependent (predecessor) tasks and 1 successor task and if any of the dependent task fails then in the fileDeletionTaskFinished() routine it will set successor tasks's success as false. Added comment in code. bq. Add a test timeout for TestDeletionService.testFileDeletionTaskDependency and remove unused imports. yeah...fixed it. bq. You have a System.out.println, must be for testing? Or you can convert it into a log message. :) yup ..for testing..moved it to debug logs. NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710369#comment-13710369
]
Hadoop QA commented on YARN-661:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12592638/YARN-661-20130716.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1504//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1504//console
This message is automatically generated.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch,
YARN-661-20130710.1.patch, YARN-661-20130711.1.patch,
YARN-661-20130712.1.patch, YARN-661-20130715.1.patch,
YARN-661-20130716.1.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710463#comment-13710463 ] Vinod Kumar Vavilapalli commented on YARN-661: -- +1. This looks good. Checking this in. NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710471#comment-13710471 ] Hudson commented on YARN-661: - SUCCESS: Integrated in Hadoop-trunk-Commit #4095 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/4095/]) YARN-661. Fixed NM to cleanup users' local directories correctly when starting up. Contributed by Omkar Vinit Joshi. (vinodkv: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1503942) * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DeletionService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/localizer/ResourceLocalizationService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestDeletionService.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Fix For: 2.1.1-beta Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch, YARN-661-20130712.1.patch, YARN-661-20130715.1.patch, YARN-661-20130716.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708735#comment-13708735
]
Hadoop QA commented on YARN-661:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12592359/YARN-661-20130715.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1480//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1480//console
This message is automatically generated.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch,
YARN-661-20130710.1.patch, YARN-661-20130711.1.patch,
YARN-661-20130712.1.patch, YARN-661-20130715.1.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707452#comment-13707452 ] Omkar Vinit Joshi commented on YARN-661: bq. FileDeletion - FileDeletionTask. Similarly all variable names from fileDeletion to fileDeletionTask. Fixed. bq. deleteHelper - scheduleFileDeletionTask Fixed. bq. fileDeletionTaskFinished should be private. fixed. bq. Also the success parameter is better captured as a field in FileDeletionTask, say succeeded. how about success? bq. It isn't doing anything for cancelling in the recursion tail. I was expecting some kind of cancel on an individual DeletionTask. I am not executing the task if its predecessor failed.. am I missing something.. didn't understand the comment. bq. We are better off tracking dependents per FileDeletionTask by having a field ListFileDeletionTask dependentTasks instead of a global map. You can add a void addDependentTasks(ListFileDeletionTask dependentTaskList), and similar getters/removers/size and you will be done. moved it to per task level and keeping it one-to-one. bq. FileDeletionTask.numberOfDependentTask is actually numberOfPendingPredecessors, right? yeah..fixed it bq. populateFileDeletionTaskDependency - addFileDeletionTaskDependencies. sure..done bq. Add proper javadoc for all public methods to help future devs. done. bq. Not related to your patch, but can you rename cleanUpFilesFromSubDir to cleanUpPerUserDirs? That's exactly what it is doing, and the current name is confusing. Similarly rename the variable names to be direct. For e.g., dirPath - userCacheDir, status - userDirStatus etc. Fixed.. makes sense. bq. Finally, can you add a specific test in TestDeletionService that validates ordering etc? Added one bq. Remove the log-message usercache path in ResourceLocalizationService, it just adds to the noise. In fact, please review all the log messages in the patch. The ones related to starting dependent task etc are also not adding much IMO. fixed. NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch, YARN-661-20130710.1.patch, YARN-661-20130711.1.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13707493#comment-13707493
]
Hadoop QA commented on YARN-661:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12592091/YARN-661-20130712.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 1
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1471//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1471//console
This message is automatically generated.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch,
YARN-661-20130710.1.patch, YARN-661-20130711.1.patch,
YARN-661-20130712.1.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13706137#comment-13706137
]
Hadoop QA commented on YARN-661:
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12591891/YARN-661-20130711.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1460//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1460//console
This message is automatically generated.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch,
YARN-661-20130710.1.patch, YARN-661-20130711.1.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705285#comment-13705285
]
Zhijie Shen commented on YARN-661:
--
I agree with the idea, and the patch looks generally good. Bellow are some
comments.
* Not sure getCancelIfAnyDependentTaskFailedFlag is necessary. If deletion of
the sub-dir fails, should deletion of the root-dir fail automatically?
* Instead of synchronizing over HashMap, how about using ConcurrentHashMap
instead?
{code}
+if (!this.deletionTaskDependencyMap.containsKey(parentTask)) {
+ this.deletionTaskDependencyMap.put(parentTask,
+new ArrayListFileDeletion());
+}
+ListFileDeletion dependentTaskList =
+this.deletionTaskDependencyMap.get(parentTask);
{code}
can be simplified as
{code}
+ListFileDeletion dependentTaskList =
+this.deletionTaskDependencyMap.putIfAbsent(
+parentTask, new ArrayListFileDeletion());
{code}
* WRT the dependency, I'd like rather call task pair predecessor and successor
instead of parent and child, because it's a dag not a tree. Moreover, how about
defining public void populateFileDeletionTaskDependency(ListFileDeletion,
FileDeletion), and populateFileDeletionTaskDependency(ListFileDeletion,
ListFileDeletion) wraps it. In fact, the former one seems to be enough for
the patch.
{code}
+ public void populateFileDeletionTaskDependency(ListFileDeletion
parentTasks,
+ ListFileDeletion childDependentTasks) {
{code}
* How about calling it delete as well, just overloading the method?
{code}
+ public void deleteHelper(FileDeletion fileDeletion) {
{code}
* Please use LocalResource.newInstance instead.
{code}
+LocalResource localResource = Records.newRecord(LocalResource.class);
{code}
* There's one finbugs warning to fix.
* In the following method
{code}
+public boolean matches(Object o) {
{code}
How about refactoring the code in the following pattern, which should be more
clear.
{code}
if (obj1 == null obj2 != null) {
return false;
} else if (obj1 != null obj2 == null) {
return false;
} else if (obj1 != null obj2 != null) {
// your logic
}
{code}
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705393#comment-13705393
]
Omkar Vinit Joshi commented on YARN-661:
Thanks [~zjshen] .. uploading updated patch..
bq. Not sure getCancelIfAnyDependentTaskFailedFlag is necessary. If deletion of
the sub-dir fails, should deletion of the root-dir fail automatically?
In this scenario we want to cancel this but may not be the case always. I will
remove it but that would have been an added control for user.
bq. Instead of synchronizing over HashMap, how about using ConcurrentHashMap
instead?
yeah fixed it..
{code}
+ListFileDeletion dependentTaskList =
+this.deletionTaskDependencyMap.putIfAbsent(
+parentTask, new ArrayListFileDeletion());
{code}
there is a problem with this as we end up creating new objects for every call.
It is present at lot of places in yarn and need to be fixed. the containsKey
check even though looks complicated avoids extra object creation on heap.
bq. WRT the dependency, I'd like rather call task pair predecessor and
successor instead of parent and child, because it's a dag not a tree. Moreover,
how about defining public void
populateFileDeletionTaskDependency(ListFileDeletion, FileDeletion), and
populateFileDeletionTaskDependency(ListFileDeletion, ListFileDeletion)
wraps it. In fact, the former one seems to be enough for the patch.
yeah renaming parent - predecessor and child - successor. I think current
(List, List) api is more flexible and we can keep it that way.
bq. How about calling it delete as well, just overloading the method?
yeah wanted to overload only but is not helping me in junit tests.
MockitTo.verify is not smart enough to distinguish between overloaded
methods...hence giving different name...
{code}
verify(delService, times(1)).deleteHelper(
argThat(new FileDeletionInclude(user, null,
new String[] { destinationFile })));
verify(delService, times(1)).deleteHelper(
argThat(new FileDeletionInclude(null, ContainerLocalizer.USERCACHE
+ _DEL_, new String[] {})));
{code}
bq. There's one finbugs warning to fix.
Hopefully fixed it.. I think it is complaining because I am exposing final
array. This too I need it for testing. Let me see if findbug complains again
then will fix it in some other way.
bq. Please use LocalResource.newInstance instead.
not related to this patch ..was just reformatting..but fixed it ..trivial fix.
bq. How about refactoring the code in the following pattern, which should be
more clear.
makes sense... fixed it..
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13705410#comment-13705410
]
Hadoop QA commented on YARN-661:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12591780/YARN-661-20130710.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1455//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/1455//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1455//console
This message is automatically generated.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch,
YARN-661-20130710.1.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13702788#comment-13702788
]
Hadoop QA commented on YARN-661:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12591342/YARN-661-20130708.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 1 new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/1434//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/1434//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/1434//console
This message is automatically generated.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.1.0-beta, 0.23.8
Reporter: Jason Lowe
Assignee: Omkar Vinit Joshi
Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13698518#comment-13698518 ] Omkar Vinit Joshi commented on YARN-661: * First part..very much straight forward.. container-executor.c already had some code to do this ...just modified ResourceLocalizationService.cleanUpFilesFromSubDir to trigger it. (basically swapping subDir with baseDir )... * I am exposing deletion task dependency to user via DeletionService. Now user can specify multilevel deletion task DAG and deletion service will take care of it one all parent (root) deletion tasks are started by user after defining dependency. I tested this locally on secured cluster.. but will add test cases to verify that DAG actually works. I will update patch with test cases attaching initial patch. NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi Attachments: YARN-661-20130701.patch YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13697277#comment-13697277 ] Omkar Vinit Joshi commented on YARN-661: taking this over... Just reproduced this issue on secured cluster..It exists.. need to be fixed.. NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[ https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13697320#comment-13697320 ] Omkar Vinit Joshi commented on YARN-661: I guess we need 2 features in deletion service. * A way for user to specify that delete all the sub directories and files inside a parent directory but don't delete parent directory. * A way to define dependency between deletion tasks. For example we need to delete usercache files before actually deleting the parent usercache itself... NM fails to cleanup local directories for users --- Key: YARN-661 URL: https://issues.apache.org/jira/browse/YARN-661 Project: Hadoop YARN Issue Type: Bug Components: nodemanager Affects Versions: 2.1.0-beta, 0.23.8 Reporter: Jason Lowe Assignee: Omkar Vinit Joshi YARN-71 added deletion of local directories on startup, but in practice it fails to delete the directories because of permission problems. The top-level usercache directory is owned by the user but is in a directory that is not writable by the user. Therefore the deletion of the user's usercache directory, as the user, fails due to lack of permissions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-661) NM fails to cleanup local directories for users
[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13653198#comment-13653198
]
Jason Lowe commented on YARN-661:
-
Sample log of the failure:
{noformat}
2013-05-09 10:38:05,107 [DeletionService #0] INFO
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Deleting
absolute path : file:/tmp/yarn-local/usercache_DEL_1368077188083/someuser
2013-05-09 10:38:05,107 [DeletionService #0] INFO
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: -- DEBUG --
deleteAsUser: [/hadoop/current/bin/container-executor, someuser, 3,
/tmp/yarn-local/usercache_DEL_1368077188083/someuser]
2013-05-09 10:38:05,142 [DeletionService #0] WARN
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Exit code
from container is : 255
2013-05-09 10:38:05,142 [DeletionService #0] ERROR
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: DeleteAsUser
for /tmp/yarn-local/usercache_DEL_1368077188083/someuser returned with non-zero
exit code255
{noformat}
The directory is owner by the user, but the parent directory which must be
written as part of the delete is owned by the mapred user and is only writable
by the mapred user.
NM fails to cleanup local directories for users
---
Key: YARN-661
URL: https://issues.apache.org/jira/browse/YARN-661
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.0.5-beta, 0.23.8
Reporter: Jason Lowe
YARN-71 added deletion of local directories on startup, but in practice it
fails to delete the directories because of permission problems. The
top-level usercache directory is owned by the user but is in a directory that
is not writable by the user. Therefore the deletion of the user's usercache
directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
