[
https://issues.apache.org/jira/browse/YARN-661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13707452#comment-13707452
]
Omkar Vinit Joshi commented on YARN-661:
----------------------------------------
bq. FileDeletion -> FileDeletionTask. Similarly all variable names from
fileDeletion to fileDeletionTask.
Fixed.
bq. deleteHelper -> scheduleFileDeletionTask
Fixed.
bq. fileDeletionTaskFinished should be private.
fixed.
bq. Also the success parameter is better captured as a field in
FileDeletionTask, say succeeded.
how about success?
bq. It isn't doing anything for cancelling in the recursion tail. I was
expecting some kind of cancel on an individual DeletionTask.
I am not executing the task if its predecessor failed.. am I missing
something.. didn't understand the comment.
bq. We are better off tracking dependents per FileDeletionTask by having a
field List<FileDeletionTask> dependentTasks instead of a global map. You can
add a void addDependentTasks(List<FileDeletionTask> dependentTaskList), and
similar getters/removers/size and you will be done.
moved it to per task level and keeping it one-to-one.
bq. FileDeletionTask.numberOfDependentTask is actually
numberOfPendingPredecessors, right?
yeah..fixed it
bq. populateFileDeletionTaskDependency -> addFileDeletionTaskDependencies.
sure..done
bq. Add proper javadoc for all public methods to help future devs.
done.
bq. Not related to your patch, but can you rename cleanUpFilesFromSubDir to
cleanUpPerUserDirs? That's exactly what it is doing, and the current name is
confusing. Similarly rename the variable names to be direct. For e.g., dirPath
-> userCacheDir, status -> userDirStatus etc.
Fixed.. makes sense.
bq. Finally, can you add a specific test in TestDeletionService that validates
ordering etc?
Added one
bq. Remove the log-message "usercache path" in ResourceLocalizationService, it
just adds to the noise. In fact, please review all the log messages in the
patch. The ones related to starting dependent task etc are also not adding much
IMO.
fixed.
> NM fails to cleanup local directories for users
> -----------------------------------------------
>
> Key: YARN-661
> URL: https://issues.apache.org/jira/browse/YARN-661
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
> Affects Versions: 2.1.0-beta, 0.23.8
> Reporter: Jason Lowe
> Assignee: Omkar Vinit Joshi
> Attachments: YARN-661-20130701.patch, YARN-661-20130708.patch,
> YARN-661-20130710.1.patch, YARN-661-20130711.1.patch
>
>
> YARN-71 added deletion of local directories on startup, but in practice it
> fails to delete the directories because of permission problems. The
> top-level usercache directory is owned by the user but is in a directory that
> is not writable by the user. Therefore the deletion of the user's usercache
> directory, as the user, fails due to lack of permissions.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
