[
https://issues.apache.org/jira/browse/MAPREDUCE-1540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12838774#action_12838774
]
Amar Kamat commented on MAPREDUCE-1540:
---------------------------------------
What actually happened was that these jobs failed during submission itself. As
a result, _job.fail()_ was invoked to cleanup the job files on the local-fs,
which inturn invokes JobTracker.finalizeJob(). JobTracker.finalizeJob() goes
ahead and adds them to _userToJobsMap_. Note that the JIP is not present in the
_jobs_ structure of the JobTracker but somehow found its way in
_userToJobsMap_. Later the _RetireJobs_ thread runs and tries to claim these
jobs. But the condition, _minConditionToRetire()_, waits until the history file
gets copied, which in this case never happens as the history file was never
created. As a result of this, the jobs will be held up in the memory forever.
Note that these jobs are not initialized and hence wont affect JobTracker's
memory.
> Sometimes JobTracker holds stale refrence of JobInProgress even after Job
> gets retired
> --------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-1540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1540
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Karam Singh
>
> Ran random writer, sort and sort validate job. Checked the jmap -histo:live
> and verified that there is no reference of JobInProgress after Jobs are
> retired
> Now submitter around 77 sleeps of around 10000 maps. then after 1 hr killed
> all the job when jobs got retired. again checked jmap -histo:live for
> JobInProgress for JT process found 2 references were there.
> Found this while doing sanity testing of 1316
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
