[
https://issues.apache.org/jira/browse/FLINK-10066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stefan Richter closed FLINK-10066.
----------------------------------
Resolution: Fixed
Merged in:
master: 160dc56fdf
release-1.6: 74323d50b0
release-1.5: 2217c09c88
> Keep only archived version of previous executions
> -------------------------------------------------
>
> Key: FLINK-10066
> URL: https://issues.apache.org/jira/browse/FLINK-10066
> Project: Flink
> Issue Type: Improvement
> Components: JobManager
> Affects Versions: 1.4.3, 1.5.2, 1.6.0
> Reporter: Stefan Richter
> Assignee: Stefan Richter
> Priority: Major
> Labels: pull-request-available
>
> Currently, the execution vertex stores a limited amount of previous
> executions in a bounded list. This happens primarily for archiving purposes
> and to remember previous locations and allocation ids. We remember the whole
> execution to eventually convert it into an archived execution.
> This seems unnecessary and dangerous as we have observed that this strategy
> is prone to memory leaks in the job manager. With a very high vertex count or
> parallelism, remembering complete executions can become very memory
> intensive. Instead I suggest to eagerly transform the executions into the
> archived version before adding them to the list, i.e. only the archived
> version is ever still referenced after the execution becomes obsolete. This
> gives better control over which information about the execution should really
> be kept in memory.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
