[jira] [Assigned] (FLINK-6626) Unifying lifecycle management of SubmittedJobGraph- and CompletedCheckpointStore
[
https://issues.apache.org/jira/browse/FLINK-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann reassigned FLINK-6626:
Assignee: (was: Tarush Grover)
> Unifying lifecycle management of SubmittedJobGraph- and
> CompletedCheckpointStore
>
>
> Key: FLINK-6626
> URL: https://issues.apache.org/jira/browse/FLINK-6626
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
>Affects Versions: 1.3.0, 1.4.0
>Reporter: Till Rohrmann
>Priority: Major
>
> Currently, Flink uses the {{SubmittedJobGraphStore}} to persist {{JobGraphs}}
> such that they can be recovered in case of failures. The
> {{SubmittedJobGraphStore}} is managed by by the {{JobManager}}. Additionally,
> Flink has the {{CompletedCheckpointStore}} which stores checkpoints for a
> given {{ExecutionGraph}}/job. The {{CompletedCheckpointStore}} is managed by
> the {{CheckpointCoordinator}}.
> The {{SubmittedJobGraphStore}} and the {{CompletedCheckpointStore}} are
> somewhat related because in the latter we store checkpoints for jobs
> contained in the former. I think it would be nice wrt lifecycle management to
> let the {{SubmittedJobGraphStore}} manage the lifecycle of the
> {{CompletedCheckpointStore}}, because often it does not make much sense to
> keep only checkpoints without a job or a job without checkpoints.
> An idea would be when we register a job with the {{SubmittedJobGraphStore}}
> then it returns a {{CompletedCheckpointStore}}. This store can then be given
> to the {{CheckpointCoordinator}} to store the checkpoints. When a job enters
> a terminal state it could be the responsibility of the
> {{SubmittedJobGraphStore}} to decide what to do with the job data
> ({{JobGraph}} and {{Checkpoints}}), e.g. keeping it or cleaning it up.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (FLINK-6626) Unifying lifecycle management of SubmittedJobGraph- and CompletedCheckpointStore
[
https://issues.apache.org/jira/browse/FLINK-6626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gary Yao reassigned FLINK-6626:
---
Assignee: Tarush Grover
> Unifying lifecycle management of SubmittedJobGraph- and
> CompletedCheckpointStore
>
>
> Key: FLINK-6626
> URL: https://issues.apache.org/jira/browse/FLINK-6626
> Project: Flink
> Issue Type: Improvement
> Components: Distributed Coordination
>Affects Versions: 1.3.0, 1.4.0
>Reporter: Till Rohrmann
>Assignee: Tarush Grover
>Priority: Major
>
> Currently, Flink uses the {{SubmittedJobGraphStore}} to persist {{JobGraphs}}
> such that they can be recovered in case of failures. The
> {{SubmittedJobGraphStore}} is managed by by the {{JobManager}}. Additionally,
> Flink has the {{CompletedCheckpointStore}} which stores checkpoints for a
> given {{ExecutionGraph}}/job. The {{CompletedCheckpointStore}} is managed by
> the {{CheckpointCoordinator}}.
> The {{SubmittedJobGraphStore}} and the {{CompletedCheckpointStore}} are
> somewhat related because in the latter we store checkpoints for jobs
> contained in the former. I think it would be nice wrt lifecycle management to
> let the {{SubmittedJobGraphStore}} manage the lifecycle of the
> {{CompletedCheckpointStore}}, because often it does not make much sense to
> keep only checkpoints without a job or a job without checkpoints.
> An idea would be when we register a job with the {{SubmittedJobGraphStore}}
> then it returns a {{CompletedCheckpointStore}}. This store can then be given
> to the {{CheckpointCoordinator}} to store the checkpoints. When a job enters
> a terminal state it could be the responsibility of the
> {{SubmittedJobGraphStore}} to decide what to do with the job data
> ({{JobGraph}} and {{Checkpoints}}), e.g. keeping it or cleaning it up.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
