[
https://issues.apache.org/jira/browse/MAPREDUCE-1699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krishna Ramachandran updated MAPREDUCE-1699:
--------------------------------------------
Attachment: mapred-1699-5.patch
Arun
I took out references to disableHistory
I do not believe we need a separate structure to store "disabled_state" for
each jobId.
In jobHistory.logSubmitted, writer (entry point for history logging) is
initialized for each jobID
If fileManager.addWriter(jobId, writer) fails, we log an error in catch block
(just like trunk)
During subsequent writing of all log events
(started/failed/killed/completed/inited/ogInfo ..) we check for
fileManager.getWriter(jobId) != null
(will be null if logSubmitted failed to initialize)
This should be sufficient and is similar to what you proposed (jobId->disabled)
I am modifying the patch accordingly
Let me know!
> JobHistory shouldn't be disabled for any reason
> -----------------------------------------------
>
> Key: MAPREDUCE-1699
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1699
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobtracker
> Affects Versions: 0.20.2
> Reporter: Arun C Murthy
> Assignee: Krishna Ramachandran
> Fix For: 0.20.3
>
> Attachments: mapred-1699-1.patch, mapred-1699-2.patch,
> mapred-1699-3.patch, mapred-1699-5.patch, mapred-1699.patch
>
>
> Recently we have had issues with JobTracker silently disabling job-history
> and starting to keep all completed jobs in memory. This leads to OOM on the
> JobTracker. We should never do this.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
