[
https://issues.apache.org/jira/browse/FLINK-9269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16457332#comment-16457332
]
ASF GitHub Bot commented on FLINK-9269:
---------------------------------------
GitHub user sihuazhou opened a pull request:
https://github.com/apache/flink/pull/5934
[FLINK-9269][state] fix concurrency problem when performing checkpoint in
HeapKeyedStateBackend
## What is the purpose of the change
*This PR fixes the concurrency problem in `HeapKeyedStateBackend`, and
contains an additional hotfix for `RocksDBKeyedStateBackend` to duplicate the
`KeySerializer` for full checkpoint*
## Brief change log
- *fix concurrency problem when performing checkpoint in
`HeapKeyedStateBackend`*
- *[hotfix] duplicate the key serializer when performing full checkpoint
in `RocksDBKeyedBackend`*
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
## Documentation
no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sihuazhou/flink FLINK-9269
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5934.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5934
----
commit f60d4d1fcd85612c792bec4d0c0d9ba1b142e1e9
Author: sihuazhou <summerleafs@...>
Date: 2018-04-28T02:50:41Z
fix concurrency problem when performing checkpoint in HeapKeyedStateBackend.
commit c998dbfc78b657bcbd47a836d23b819ce61778b9
Author: sihuazhou <summerleafs@...>
Date: 2018-04-28T02:57:11Z
[hotfix] duplicate the key serializer when performing full checkpoint in
RocksDBKeyedBackend.
----
> Concurrency problem in HeapKeyedStateBackend when performing checkpoint async
> -----------------------------------------------------------------------------
>
> Key: FLINK-9269
> URL: https://issues.apache.org/jira/browse/FLINK-9269
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.5.0
> Reporter: Sihua Zhou
> Assignee: Sihua Zhou
> Priority: Blocker
> Fix For: 1.5.0
>
>
> {code:java}
> @Nonnull
> @Override
> protected SnapshotResult<KeyedStateHandle> performOperation() throws
> Exception {
> // do something
> long[] keyGroupRangeOffsets = new
> long[keyGroupRange.getNumberOfKeyGroups()];
> for (int keyGroupPos = 0; keyGroupPos <
> keyGroupRange.getNumberOfKeyGroups(); ++keyGroupPos) {
> int keyGroupId = keyGroupRange.getKeyGroupId(keyGroupPos);
> keyGroupRangeOffsets[keyGroupPos] = localStream.getPos();
> outView.writeInt(keyGroupId);
> for (Map.Entry<String, StateTable<K, ?, ?>> kvState :
> stateTables.entrySet()) {
> // do something
> }
> }
> // do something
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
