[
https://issues.apache.org/jira/browse/FLINK-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15932078#comment-15932078
]
ASF GitHub Bot commented on FLINK-6014:
---------------------------------------
Github user shixiaogang commented on the issue:
https://github.com/apache/flink/pull/3524
@StephanEwen Thanks a lot for your valuable comments. I will update the PR
as suggested.
* I think it's a good idea that we make the `StateRegistry` into
`SharedStateRegistry`. That will need (1) the state handle must not discard
registered objects in the `discardState` method and (2) the state handle has to
register shared object once it is received by the coordinator (now the state
handle does not register their objects before its checkpoint completes).
* Now that state handles have to register their objects once they are
received by the coordinator, we should move `SharedStateRegister` from
`CompletedCheckpointStore` to `CheckpointCoordinator`.
* It's better for `StateRegistry` directly discard an object once its
reference count is 0. I used a list to collect discarded objects because I want
to make the changes in the discarding of completed checkpoint as few as
possible.
> Allow the registration of state objects in checkpoints
> ------------------------------------------------------
>
> Key: FLINK-6014
> URL: https://issues.apache.org/jira/browse/FLINK-6014
> Project: Flink
> Issue Type: Sub-task
> Components: State Backends, Checkpointing
> Reporter: Xiaogang Shi
> Assignee: Xiaogang Shi
>
> This issue is the very first step towards incremental checkpointing. We
> introduce a new state handle named {{CompositeStateHandle}} to be the base of
> the snapshots taken by task components. Known implementation may include
> {{KeyedStateHandle}} (for {{KeyedStateBackend}}s), {{SubtaskState}} (for
> subtasks, splits of {{JobVertex}}) and {{TaskState}} (for {{JobVertex}}s).
> Each {{CompositeStateHandle}} is composed of a collection of {{StateObject}s.
> It should register all its state objects in {{StateRegistry}} when its
> checkpoint is added into {{CompletedCheckpointStore}} (i.e., a pending
> checkpoint completes or a complete checkpoint is reloaded in the recovery).
> When a completed checkpoint is moved out of the {{CompletedCheckpointStore}},
> we should not simply discard all state objects in the checkpoint. With the
> introduction of incremental checkpointing, a {{StateObject}} may be
> referenced by different checkpoints. We should unregister all the state
> objects contained in the {{StateRegistry}} first. Only those state objects
> that are not referenced by any checkpoint can be deleted.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
