[
https://issues.apache.org/jira/browse/HADOOP-5994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amar Kamat updated HADOOP-5994:
-------------------------------
Attachment: HADOOP-5994-v1.7.patch
Attaching a patch that moves completed jobs to a _done_ subfolder. Changes the
testcases accordingly. Following are the changes
# jobhistory now maintains an internal mapping from job-id to
jobhistory-filename
# finalize recovery reuses these cached values for renaming
# jobtracker now invoked jobhistory.markAsCompleted() which moves the job to
the done folder. the config files are also moved to the done folder
# jobhistory's searching api's namely getJobHistoryFilename() and
getJobHistoryLogLocation() now expect a boolean indicating if the job is
runnning or complete
# added a check in TestJobHistory to check if the filename returned by the
framework after job completion belongs to a done folder or not, files are
removed from the running folder. Same is also tested for config files too
# HistoryCleaner now runs on done folder
# Made sure that once the job files are moved to done folder, the framework
(except historycleaner) is not affected by changes to done folder. History
cleaner will fail and re-run upon an IOException.
Running test-patch
Tested this patch on a single node cluster and it works fine.
> Move the completed jobs' history files to a DONE subdirectory inside the
> configured history directory
> -----------------------------------------------------------------------------------------------------
>
> Key: HADOOP-5994
> URL: https://issues.apache.org/jira/browse/HADOOP-5994
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Reporter: Devaraj Das
> Assignee: Amar Kamat
> Fix For: 0.21.0
>
> Attachments: HADOOP-5994-v1.7.patch
>
>
> Whenever a job completes, the history file can be moved to a directory called
> DONE. That would make the management of job history files easier (for
> example, administrators can move the history files from that directory to
> some other place, delete them, archive them, etc.).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
