[
https://issues.apache.org/jira/browse/FLINK-8836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444041#comment-16444041
]
ASF GitHub Bot commented on FLINK-8836:
---------------------------------------
GitHub user StefanRRichter opened a pull request:
https://github.com/apache/flink/pull/5880
[FLINK-8836] Fix duplicate method in KryoSerializer to perform deep c…
…opy of default/registered serializer instances.
This method did create deep copies of registered or default serializer
instances and
as a result those serializer instances can accidentally be shared across
different threads.
## Brief change log
This PR fixes a problem with the `duplicate` method of `KryoSerializer`. We
do now perform deep copies for default and registered serializer objects.
Otherwise, if we share duplicated `KryoSerializer` instances across
different threads, some of their internal serializers might be a shared
instance of a stateful object.
## Verifying this change
This change added tests and can be verified by running
`KryoSerializerConcurrencyTest`.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): (no)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: (no)
- The serializers: (yes)
- The runtime per-record code paths (performance sensitive): (no)
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Yarn/Mesos, ZooKeeper: (indirectly)
- The S3 file system connector: (no)
## Documentation
- Does this pull request introduce a new feature? ( no)
- If yes, how is the feature documented? (not applicable)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/StefanRRichter/flink FLINK-8836
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5880.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5880
----
commit 617ebd114b86d8f00d15e53dd649bb633fc717f9
Author: Stefan Richter <s.richter@...>
Date: 2018-04-19T13:10:07Z
[FLINK-8836] Fix duplicate method in KryoSerializer to perform deep copy of
default/registered serializer instances.
This method did create deep copies of registered or default serializer
instances and
as a result those serializer instances can accidentally be shared across
different threads.
----
> Duplicating a KryoSerializer does not duplicate registered default serializers
> ------------------------------------------------------------------------------
>
> Key: FLINK-8836
> URL: https://issues.apache.org/jira/browse/FLINK-8836
> Project: Flink
> Issue Type: Bug
> Components: Type Serialization System
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Stefan Richter
> Priority: Blocker
> Fix For: 1.5.0
>
>
> The {{duplicate()}} method of the {{KryoSerializer}} is as following:
> {code:java}
> public KryoSerializer<T> duplicate() {
> return new KryoSerializer<>(this);
> }
> protected KryoSerializer(KryoSerializer<T> toCopy) {
> defaultSerializers = toCopy.defaultSerializers;
> defaultSerializerClasses = toCopy.defaultSerializerClasses;
> kryoRegistrations = toCopy.kryoRegistrations;
> ...
> }
> {code}
> Shortly put, when duplicating a {{KryoSerializer}}, the
> {{defaultSerializers}} serializer instances are directly provided to the new
> {{KryoSerializer}} instance.
> This causes the fact that those default serializers are shared across two
> different {{KryoSerializer}} instances, and therefore not a correct duplicate.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
