[
https://issues.apache.org/jira/browse/MAPREDUCE-721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728581#action_12728581
]
Amar Kamat commented on MAPREDUCE-721:
--------------------------------------
So here is what might have happened. Consider the following sequence of events
||Time||Event||
|x|delete threads runs once|
|x + 20| job completes|
|x + 60 | delete thread runs and finds that job has completed 40 mins so
ignores it|
|x + 120| delete thread runs and deletes the job status|
So from the time the job completed, its {{(x + 120) - (x + 20) = 1hr 40mins}}.
We should decrease the check interval.
> When "mapred.job.tracker.persist.jobstatus.hours" is set to x hours, the
> files are not getting deleted after that x hours.It gets deleted after some
> random hours which is greater than x hours.
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-721
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-721
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Iyappan Srinivasan
>
> When "mapred.job.tracker.persist.jobstatus.hours" is set to 1 hour, the jobid
> files, present in configured directory, are not getting deleted after that 1
> hour. It gets deleted after some random hours.
> This is irrespective of whether "mapred.job.tracker.persist.jobstatus.dir" is
> set to local file system or DFS.
> Steps to reproduce the issue:
> 1) Im mapred-site.xml,
> Set "mapred.job.tracker.persist.jobstatus.active" to "true"
> Set "mapred.job.tracker.persist.jobstatus.hours" to "1"
> Set "mapred.job.tracker.persist.jobstatus.dir" to either a local file syste
> by preixing it with file:/// or dfs by giving a dfs directory without the
> "file:///"
> 2) After that run any job.
> 3) After it is complete the job id file comes in the configured directory.
> 4) Wait for one hour
> 5) Observe After 1 hour the files are not deleted.
> 6) After some random hours, files gets deleted.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
