[
https://issues.apache.org/jira/browse/BEAM-10375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17169011#comment-17169011
]
Beam JIRA Bot commented on BEAM-10375:
--------------------------------------
This issue is assigned but has not received an update in 30 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> HBaseIO.Read not deterministic coder exception when executed with flink 1.10.1
> ------------------------------------------------------------------------------
>
> Key: BEAM-10375
> URL: https://issues.apache.org/jira/browse/BEAM-10375
> Project: Beam
> Issue Type: Bug
> Components: io-java-hbase
> Affects Versions: 2.21.0
> Reporter: Richard Moorhead
> Assignee: Ismaël Mejía
> Priority: P2
> Labels: stale-assigned
>
> Given that HBaseIO.readAll() is applied to a PCollection<HBaseIO.Read>, the
> following exception is thrown when readAll is executed by the flink runner:
> {code:java}
> The program finished with the following exception:
> org.apache.flink.client.program.ProgramInvocationException: The main method
> caused an error: the keyCoder of a GroupByKey must be deterministic
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:335)
> at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:205)
> at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:138)
> at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:662)
> at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:210)
> at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:893)
> at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:966)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
> at
> org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
> at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:966)
> Caused by: java.lang.IllegalStateException: the keyCoder of a GroupByKey must
> be deterministic
> at
> org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:234)
> at
> org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:110)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:476)
> at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:355)
> at org.apache.beam.sdk.transforms.Reshuffle.expand(Reshuffle.java:87)
> at org.apache.beam.sdk.transforms.Reshuffle.expand(Reshuffle.java:49)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:542)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
> at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:368)
> at
> org.apache.beam.runners.core.construction.SplittableParDoNaiveBounded$SplittableProcessNaive.expand(SplittableParDoNaiveBounded.java:
> 110)
> at
> org.apache.beam.runners.core.construction.SplittableParDoNaiveBounded$SplittableProcessNaive.expand(SplittableParDoNaiveBounded.java:
> 95)
> at org.apache.beam.sdk.Pipeline.applyReplacement(Pipeline.java:569)
> at org.apache.beam.sdk.Pipeline.replace(Pipeline.java:294)
> at org.apache.beam.sdk.Pipeline.replaceAll(Pipeline.java:212)
> at
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.translate(FlinkPipelineExecutionEnvironment.java:114)
> at org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:107)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:317)
> at org.apache.beam.sdk.Pipeline.run(Pipeline.java:303)
> at
> com.cerner.pophealth.mpm.processing.beam.pipeline.bounded.EmpiPersonConversionPipeline.main(EmpiPersonConversionPipeline.java:96)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:321)
> ... 11 more
> Caused by: org.apache.beam.sdk.coders.Coder$NonDeterministicException:
> SerializableCoder(org.apache.beam.sdk.io.hbase.HBaseIO$Read) is not deter
> ministic because:
> Java Serialization may be non-deterministic.
> at
> org.apache.beam.sdk.coders.SerializableCoder.verifyDeterministic(SerializableCoder.java:205)
> at
> org.apache.beam.sdk.transforms.GroupByKey.expand(GroupByKey.java:232) {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
