[
https://issues.apache.org/jira/browse/MAPREDUCE-896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ravi Gummadi updated MAPREDUCE-896:
-----------------------------------
Attachment: MR-896.patch
Attaching patch with the fix. Please review and provide your comments.
Patch does the following:
(1) When deleting $jobId/$attemptId/work or when deleting $jobId/$attemptId, TT
uses task-controller to enable the path for deletion(by changing permissions).
(a) LinuxTaskController sets 770 as permissions for all the files/directories
within this dir recursively and then TT will delete the dir. (b)
DefaultTaskController sets rwx for user(same as TT) for this dir recursively
and then TT deletes the dir.
(2) Deletion of $jobId is done as earlier because user can't create any files
in this dir.
(3) Deletion of work dir in TaskRunner(useful with jvm reuse) is also done by
changing the permissions first(no taskcontroller is needed in this case).
(4) With jvm reuse, after final task of a Jvm is finished, workDir is deleted
using procedure mentioned in (1) above by task-controller.
(5) Modified TestTaskTrackerLocalization and
TestLocalizationWithLinuxTaskController to have directories/files created in
$attemptId and set nonwritable permissions which tests whether (a)
DefaultTaskController and (b) LinuxTaskController would be able to remove these
new files from $attemptId.
> Users can set non-writable permissions on temporary files for TT and can
> abuse disk usage.
> ------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-896
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-896
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Affects Versions: 0.21.0
> Reporter: Vinod K V
> Assignee: Ravi Gummadi
> Fix For: 0.21.0
>
> Attachments: MR-896.patch
>
>
> As of now, irrespective of the TaskController in use, TT itself does a full
> delete on local files created by itself or job tasks. This step, depending
> upon TT's umask and the permissions set by files by the user, for e.g in
> job-work/task-work or child.tmp directories, may or may not go through
> successful completion fully. Thus is left an opportunity for abusing disk
> space usage either accidentally or intentionally by TT/users.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
