Kenneth Knowles created BEAM-4689:
-------------------------------------
Summary: Dataflow postcommit broken
Key: BEAM-4689
URL: https://issues.apache.org/jira/browse/BEAM-4689
Project: Beam
Issue Type: New Feature
Components: runner-dataflow
Reporter: Kenneth Knowles
Assignee: Thomas Groh
The Dataflow postcommit is broken in a way that seems real and user-impacting:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/testReport/junit/org.apache.beam.sdk.transforms/SplittableDoFnTest/testSideInput/
{code}
Caused by: java.lang.IllegalArgumentException: unable to deserialize Serialized
DoFnInfo
...
Caused by: java.io.InvalidClassException:
org.apache.beam.runners.core.construction.SplittableParDo$RandomUniqueKeyFn;
local class incompatible: stream classdesc serialVersionUID =
6068396661487412884, local class serialVersionUID = -617521663543732196
{code}
This means that the worker is using a version of the class from its own
classpath, not the version from the user's staged pipeline. It implies that the
worker is not shading runners-core-construction. Because that is where a ton of
utility DoFns live, it is critical that it be shaded.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
