GitHub user StefanRRichter opened a pull request:
https://github.com/apache/flink/pull/4745
[FLINK-7720] [state backends] Centralize creation of backends and state
related resources
## What is the purpose of the change
This PR combines [FLINK-7719] and [FLINK-7720], which are both related to
our efforts to introduce task recovery from local state. [FLINK-7719] makes the
checkpoint id part of the task deployment descriptior, which is required so
that tasks can figure our the right state for local recovery. [FLINK-7720]
introduces state manager components on task manager, task, and stream task
level that build upon each other. The purpose of them is unifying the creation
of state related elements such as backends. State reporting now goes through
those state managers.
## Brief change log
[FLINK-7719]
- *JobManagerTaskRestore carries the checkpoint state handles and
checkpoint id that the JM wants the task to restore*
[FLINK-7720]
- *Introduced TaskExecutorLocalStateStoresManager (mostly as
placeholder/mock), TaskLocalStateStore (currently only a placeholder/ mock),
TaskStateManager, and StreamTaskStateManager. Those build a hierarchy for state
reporting and recovery.*
- *All state related elements are now created by the
StreamTaskStateManager and owned by the stream operator.*
## Verifying this change
This change is already covered by existing tests and is currently mostly a
refactoring and preparation for future extension. New test cases have been
added for implementations of new components. There should currently no
observable changes for users.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (yes)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
## Documentation
- Does this pull request introduce a new feature? (no)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StefanRRichter/flink state-manager-pull
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4745.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4745
----
commit e20958a2b54856cba353ea52bc603a16bee0611c
Author: Stefan Richter <[email protected]>
Date: 2017-08-21T12:31:38Z
[FLINK-7719] [checkpoints] Send checkpoint id to task as part of deployment
descriptor when resuming
commit c27c692a36305c79329e01031a43ae880039c9fd
Author: Stefan Richter <[email protected]>
Date: 2017-09-27T13:10:52Z
[FLINK-7720] [checkpoints] Centralize creation of backends and state
related resources
----
---
