[ http://issues.apache.org/jira/browse/HADOOP-381?page=all ]
Owen O'Malley updated HADOOP-381:
---------------------------------
Attachment: keep-task-file-pattern.patch
1. adds the keep.failed.task.files and keep.task.files.pattern variables to
hadoop-default.xml
2. adds set/getKeepTaskFilesPattern to JobConf
3. modifies the TaskTracker to keep the files at Task completion if the task
name matches the regex.
4. adds a word count test to the junit tests.
5. adds a test to make sure that task trackers clean up normally
6. also checks that the keep.task.files.pattern keeps the directories
7. the mini-mr cluster keeps track of the local dir for each task tracker
8. the mini-mr cluster adds a waitUntilIdle() method that waits for the cluster
to be stable
9. task tracker adds isIdle to test whether a given task tracker is idle
10. add the examples to the classpath for the junit tests
> keeping files for tasks that match regex on task id
> ---------------------------------------------------
>
> Key: HADOOP-381
> URL: http://issues.apache.org/jira/browse/HADOOP-381
> Project: Hadoop
> Issue Type: New Feature
> Components: mapred
> Affects Versions: 0.4.0
> Reporter: Owen O'Malley
> Assigned To: Owen O'Malley
> Fix For: 0.5.0
>
> Attachments: keep-task-file-pattern.patch
>
>
> For debugging map/reduce jobs, if a single task is producing bad results, but
> *not* failing, it is hard to debug the problem. This patch lets you set a
> pattern for task ids that will keep their files from being deleted when the
> task and job complete. This allows the developer to run the task in the
> IsolationRunner under the debugger.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
