GitHub user tillrohrmann opened a pull request:
https://github.com/apache/flink/pull/1818
[FLINK-3633] Fix user code de/serialization in ExecutionConfig
FLINK-3327 moved the ExecutionConfig directly to the JobGraph so that it
was serialized
and deserialized using the system class loader when sending a SubmitJob
message to the
JobManager. This is problematic since the ExecutionConfig can contain user
code class
which require a user code class loader for deserialization. In order to
circumvent the
problem, a UserCodeValue class was introduced which automatically sends the
wrapped value
as a byte array. On the receiving side, the wrapped value has to be
explicitly deserialized
providing a class loader.
To test the feature the ScalaShellITCase.testSubmissionOfExternalLibrary
was adapted
to register org.apache.flink.ml.math.Vector at the ExecutionConfig.
This commit also re-introduces the removed ExecutionConfig.CONFIG_KEY key,
so that
version 1.1 does not break the API.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tillrohrmann/flink fixJobSubmission
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/1818.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1818
----
commit 7abef1b685d76a5249ddeb0fcc4f1190b541ba57
Author: Till Rohrmann <[email protected]>
Date: 2016-03-18T15:22:04Z
[FLINK-3633] Fix user code de/serialization in ExecutionConfig
FLINK-3327 moved the ExecutionConfig directly to the JobGraph so that it
was serialized
and deserialized using the system class loader when sending a SubmitJob
message to the
JobManager. This is problematic since the ExecutionConfig can contain user
code class
which require a user code class loader for deserialization. In order to
circumvent the
problem, a UserCodeValue class was introduced which automatically sends the
wrapped value
as a byte array. On the receiving side, the wrapped value has to be
explicitly deserialized
providing a class loader.
To test the feature the ScalaShellITCase.testSubmissionOfExternalLibrary
was adapted
to register org.apache.flink.ml.math.Vector at the ExecutionConfig.
This commit also re-introduces the removed ExecutionConfig.CONFIG_KEY key,
so that
version 1.1 does not break the API.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
