[
https://issues.apache.org/jira/browse/FLINK-6025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15906552#comment-15906552
]
ASF GitHub Bot commented on FLINK-6025:
---------------------------------------
GitHub user tzulitai opened a pull request:
https://github.com/apache/flink/pull/3517
[FLINK-6025] [core] Add Flink's own JavaSerializer for Kryo serialization
(This PR should also be backported for `release-1.1` and `release-1.2`.
Separate PRs are not opened for the backports because it is a relatively
self-contained change.)
This PR adds a reimplemented `JavaSerializer` to be registered with
Kryo. This is due to a know issue with Kryo's `JavaSerializer` that may
use the wrong classloader for deserialzation (see
https://github.com/EsotericSoftware/kryo/pull/483).
Since we by default register Kryo's `JavaSerializer` for `Throwable`s,
users who need to serialize exception types as messages bump into
`ClassNotFoundException`s.
Instead of registering Kryo's `JavaSerializer` for `Throwables`, it is now
changed to register the reimplemented `JavaSerializer`.
Generally, users who bump into `ClassNotFoundException`s if they are using
Kryo's `JavaSerializer` for
their own custom types are also recommended to change to Flink's
`JavaSerializer`.
The change is tested using the minimal issue reproducing example provided
by our users in
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tzulitai/flink FLINK-6025
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/3517.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3517
----
commit 0e85ec1bf0a11c67e2af3625e0858e22b41600cd
Author: Tzu-Li (Gordon) Tai <[email protected]>
Date: 2017-03-12T14:46:27Z
[FLINK-6025] [core] Add Flink's own JavaSerializer for Kryo serialization
This commit adds a reimplemented JavaSerializer to be registered with
Kryo. This is due to a know issue with Kryo's JavaSerializer that may
use the wrong classloader for deserialzation.
Instead of registering Kryo's JavaSerializer for Throwables, it is now
changed to register the reimplemented JavaSerializer. Users who bump
into ClassNotFoundExceptions if they are using Kryo's JavaSerializer for
their own types are also recommended to change to Flink's JavaSerializer.
----
> User code ClassLoader not used when KryoSerializer fallbacks to serialization
> for copying
> -----------------------------------------------------------------------------------------
>
> Key: FLINK-6025
> URL: https://issues.apache.org/jira/browse/FLINK-6025
> Project: Flink
> Issue Type: Bug
> Components: Type Serialization System
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
>
> Reported in ML:
> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/AWS-exception-serialization-problem-td12063.html
> This is caused by a known Kryo issue with its {{JavaSerializer}}:
> https://github.com/EsotericSoftware/kryo/pull/483.
> It happens when a {{Throwable}} is to be copied by the {{KryoSerialzer}}.
> Since we use the {{JavaSerializer}} for throwables, and {{JavaSerializer}}
> doesn't support copying, the {{KryoSerializer}} fallbacks to use
> de-/serialization for the throwable. The problem is that on deserialization,
> the classloader that the {{ObjectInputStream}} uses may be overriden, and
> doesn't specifically uses Kryo's configured classloader (i.e., the user code
> class loader), and results in {{ClassNotFoundException}}.
> Generally, this may happen if the user also registers to use the
> {{JavaSerializer}} for their types.
> To fix the problem for {{Throwable}} serializing in the {{KryoSerializer}},
> we could either consider registering our own fixed {{JavaSerializer}} for
> throwables, or wait for the Kryo fix to be released (to be fixed in Kryo
> 4.0.1 release).
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
