Jason Lowe created MAPREDUCE-4705:
-------------------------------------
Summary: Historyserver links expire before the history data does
Key: MAPREDUCE-4705
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4705
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobhistoryserver, mrv2
Affects Versions: 0.23.3
Reporter: Jason Lowe
Priority: Critical
The historyserver can serve up links to jobs that become useless well before
the job history files are purged. For example on a large, heavily used cluster
we can end up rotating through the maximum number of jobs the historyserver can
track fairly quickly. If a user was investigating an issue with a job using a
saved historyserver URL, that URL can become useless because the historyserver
has forgotten about the job even though the history files are still sitting in
HDFS.
We can tell the historyserver to keep track of more jobs by increasing
{{mapreduce.jobhistory.joblist.cache.size}}, but this has a direct impact on
the responsiveness of the main historyserver page since it serves up all the
entries to the client at once. It looks like Hadoop 1.x avoided this issue by
encoding the history file location into the URLs served up by the
historyserver, so it didn't have to track a mapping between job ID and history
file location.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
