[
https://issues.apache.org/jira/browse/FLINK-9263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16456318#comment-16456318
]
ASF GitHub Bot commented on FLINK-9263:
---------------------------------------
GitHub user sihuazhou opened a pull request:
https://github.com/apache/flink/pull/5930
[FLINK-9263][state] Fix concurrency problem in DefaultOperatorStateBackend.
## What is the purpose of the change
This PR fixes the concurrency problem in `DefaultOperatorStateBackend`, the
reason for this bug is that when performing checkpoint we didn't create a true
`deepCopy` of the `registeredOperatorStates` and `registeredBroadcastStates`.
## Brief change log
- *create a truly deep copy of `registeredBroadcastStates` and
`registeredOperatorStates` when performing checkpoint*
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (no)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (yes)
- The S3 file system connector: (no)
## Documentation
no
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/sihuazhou/flink
fixConcurrencyProblemInOperatorStateBackend
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5930.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5930
----
commit 93aaa53ddb43f644fa572aedd914159f88516c84
Author: sihuazhou <summerleafs@...>
Date: 2018-04-27T12:42:33Z
fix concurrency problem in DefaultOperatorStateBackend.
----
> Kafka010ITCase failed on travis flaky
> -------------------------------------
>
> Key: FLINK-9263
> URL: https://issues.apache.org/jira/browse/FLINK-9263
> Project: Flink
> Issue Type: Bug
> Components: Tests
> Affects Versions: 1.5.0
> Reporter: Sihua Zhou
> Assignee: Sihua Zhou
> Priority: Blocker
> Fix For: 1.5.0
>
>
> instance: https://travis-ci.org/apache/flink/jobs/371952726
> I found this
> {code}
> Caused by: java.util.concurrent.ExecutionException:
> java.lang.IllegalStateException: Concurrent access to KryoSerializer. Thread
> 1: pool-26-thread-2 , Thread 2: pool-26-thread-1
> at java.util.concurrent.FutureTask.report(FutureTask.java:122)
> at java.util.concurrent.FutureTask.get(FutureTask.java:192)
> at
> org.apache.flink.util.FutureUtil.runIfNotDoneAndGet(FutureUtil.java:53)
> at
> org.apache.flink.streaming.api.operators.OperatorSnapshotFinalizer.<init>(OperatorSnapshotFinalizer.java:53)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask$AsyncCheckpointRunnable.run(StreamTask.java:858)
> ... 5 more
> Caused by: java.lang.IllegalStateException: Concurrent access to
> KryoSerializer. Thread 1: pool-26-thread-2 , Thread 2: pool-26-thread-1
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.enterExclusiveThread(KryoSerializer.java:622)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.serialize(KryoSerializer.java:254)
> at
> org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:125)
> at
> org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
> at
> org.apache.flink.runtime.state.DefaultOperatorStateBackend$PartitionableListState.write(DefaultOperatorStateBackend.java:687)
> at
> org.apache.flink.runtime.state.DefaultOperatorStateBackend$1.performOperation(DefaultOperatorStateBackend.java:423)
> at
> org.apache.flink.runtime.state.DefaultOperatorStateBackend$1.performOperation(DefaultOperatorStateBackend.java:352)
> at
> org.apache.flink.runtime.io.async.AbstractAsyncCallableWithResources.call(AbstractAsyncCallableWithResources.java:75)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.flink.util.FutureUtil.runIfNotDoneAndGet(FutureUtil.java:50)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
