Maximilian Michels created FLINK-1843:
-----------------------------------------
Summary: Job History gets cleared too fast
Key: FLINK-1843
URL: https://issues.apache.org/jira/browse/FLINK-1843
Project: Flink
Issue Type: Bug
Components: JobManager
Affects Versions: 0.9
Reporter: Maximilian Michels
Fix For: 0.9
As per FLINK-1442, the JobManager stores the archived ExecutionGraph behind a
SoftReference. At least for local setups, this mechanism doesn't seem to work
properly. There are two issues:
- The history gets cleared too fast
- The history gets cleared in a non-sequential fashion, i.e. arbitrary old
ExecutionGraph are discarded
To solve these problems we might
- Store the least recent ExecutionGraph behind a SoftReference
- Store the most recent ExecutionGraphs without a SoftReference
That way, we can save memory but have the latest history available to the user.
We might introduce a configuration variable where the user can specify the
number of ExecutionGraphs that should be held in memory. The remaining can be
stored behind a SoftReference.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
