[jira] [Commented] (BEAM-6884) NoSuchMethodError: descriptors$EnumValueDescriptor when deploying Beam Java SDK 2.10.0 to Dataflow
[
https://issues.apache.org/jira/browse/BEAM-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17122883#comment-17122883
]
Beam JIRA Bot commented on BEAM-6884:
-
This issue is P2 but has been unassigned without any comment for 60 days so it
has been labeled "stale-P2". If this issue is still affecting you, we care!
Please comment and remove the label. Otherwise, in 14 days the issue will be
moved to P3.
Please see https://beam.apache.org/contribute/jira-priorities/ for a detailed
explanation of what these priorities mean.
> NoSuchMethodError: descriptors$EnumValueDescriptor when deploying Beam Java
> SDK 2.10.0 to Dataflow
> --
>
> Key: BEAM-6884
> URL: https://issues.apache.org/jira/browse/BEAM-6884
> Project: Beam
> Issue Type: Bug
> Components: beam-model
>Affects Versions: 2.10.0
>Reporter: Yohei Shimomae
>Priority: P2
> Labels: stale-P2
>
> My working environment:
> * Apache Beam Java SDK version: works with 2.9.0 but failed with 2.10.0
> * Runner: failed with both Direct Runner and Dataflow Runner
> * Application code: Scala (note I did not use Scio)
> I tried to change Apache Beam Java SDK version from 2.9.0 to 2.10.0 and
> deploy it to Dataflow but I got this error. It works with 2.9.0. Am I missing
> something?
> {code:java}
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.beam.model.pipeline.v1.RunnerApi$BeamConstants$Constants.getValueDescriptor()Lorg/apache/beam/vendor/grpc/v1p13p1/com/google/protobuf/Descriptors$EnumValueDescriptor;
> at
> org.apache.beam.sdk.transforms.windowing.BoundedWindow.extractTimestampFromProto(BoundedWindow.java:84)
> at
> org.apache.beam.sdk.transforms.windowing.BoundedWindow.(BoundedWindow.java:49)
> at
> org.apache.beam.sdk.coders.CoderRegistry$CommonTypes.(CoderRegistry.java:140)
> at
> org.apache.beam.sdk.coders.CoderRegistry$CommonTypes.(CoderRegistry.java:97)
> at org.apache.beam.sdk.coders.CoderRegistry.(CoderRegistry.java:160)
> at org.apache.beam.sdk.Pipeline.getCoderRegistry(Pipeline.java:326)
> at org.apache.beam.sdk.io.kafka.KafkaIO$Read.expand(KafkaIO.java:707)
> at org.apache.beam.sdk.io.kafka.KafkaIO$Read.expand(KafkaIO.java:309)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
> {code}
> My code is in Scala but it works well with Beam 2.9.0.
> {code:java}
> val p = Pipeline.create(options)
> p.apply(s"${bu.name}_ReadFromKafka", KafkaIO.read()
> .withBootstrapServers(options.getBootstreapServers)
> .updateConsumerProperties(config)
> .withTopics(util.Arrays.asList(topicName))
> .withKeyDeserializer(classOf[LongDeserializer])
> .withValueDeserializer(classOf[StringDeserializer])
> .withConsumerFactoryFn(
> new KafkaTLSConsumerFactory(
> projectId, options.getSourceBucket, options.getTrustStoreGCSKey,
> options.getKeyStoreGCSKey)))
> .apply(s"${bu.name}_Convert", ParDo.of(new
> ConvertJSONTextToEPCTransaction(bu)))
> .apply(s"${bu.name}_WriteToBQ", BigQueryIO.write()
> .to(bqDestTable)
> .withSchema(schema)
> .withFormatFunction(new ConvertMessageToTable())
> .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_NEVER)
> .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND))
> }
> p.run
> {code}
> According to the error log, it failed at this part.
>
> [https://github.com/apache/beam/blob/v2.10.0/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/BoundedWindow.java#L81-L85]
> {code:java}
> private static Instant
> extractTimestampFromProto(RunnerApi.BeamConstants.Constants constant) {
> return new Instant(
> Long.parseLong(
>
> constant.getValueDescriptor().getOptions().getExtension(RunnerApi.beamConstant)));
> }
> {code}
> This constant come from this part.
>
> [https://github.com/apache/beam/blob/v2.10.0/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/BoundedWindow.java#L48-L49]
> {code:java}
> public static final Instant TIMESTAMP_MIN_VALUE =
>
> extractTimestampFromProto(RunnerApi.BeamConstants.Constants.MIN_TIMESTAMP_MILLIS);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (BEAM-6884) NoSuchMethodError: descriptors$EnumValueDescriptor when deploying Beam Java SDK 2.10.0 to Dataflow
[
https://issues.apache.org/jira/browse/BEAM-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947098#comment-16947098
]
Kirill Kozlov commented on BEAM-6884:
-
I had similar issue when running tests using Intellij, running the same test
with Gradle works.
> NoSuchMethodError: descriptors$EnumValueDescriptor when deploying Beam Java
> SDK 2.10.0 to Dataflow
> --
>
> Key: BEAM-6884
> URL: https://issues.apache.org/jira/browse/BEAM-6884
> Project: Beam
> Issue Type: Bug
> Components: beam-model
>Affects Versions: 2.10.0
>Reporter: Yohei Shimomae
>Priority: Major
>
> My working environment:
> * Apache Beam Java SDK version: works with 2.9.0 but failed with 2.10.0
> * Runner: failed with both Direct Runner and Dataflow Runner
> * Application code: Scala (note I did not use Scio)
> I tried to change Apache Beam Java SDK version from 2.9.0 to 2.10.0 and
> deploy it to Dataflow but I got this error. It works with 2.9.0. Am I missing
> something?
> {code:java}
> Exception in thread "main" java.lang.NoSuchMethodError:
> org.apache.beam.model.pipeline.v1.RunnerApi$BeamConstants$Constants.getValueDescriptor()Lorg/apache/beam/vendor/grpc/v1p13p1/com/google/protobuf/Descriptors$EnumValueDescriptor;
> at
> org.apache.beam.sdk.transforms.windowing.BoundedWindow.extractTimestampFromProto(BoundedWindow.java:84)
> at
> org.apache.beam.sdk.transforms.windowing.BoundedWindow.(BoundedWindow.java:49)
> at
> org.apache.beam.sdk.coders.CoderRegistry$CommonTypes.(CoderRegistry.java:140)
> at
> org.apache.beam.sdk.coders.CoderRegistry$CommonTypes.(CoderRegistry.java:97)
> at org.apache.beam.sdk.coders.CoderRegistry.(CoderRegistry.java:160)
> at org.apache.beam.sdk.Pipeline.getCoderRegistry(Pipeline.java:326)
> at org.apache.beam.sdk.io.kafka.KafkaIO$Read.expand(KafkaIO.java:707)
> at org.apache.beam.sdk.io.kafka.KafkaIO$Read.expand(KafkaIO.java:309)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:488)
> at org.apache.beam.sdk.values.PBegin.apply(PBegin.java:56)
> at org.apache.beam.sdk.Pipeline.apply(Pipeline.java:182)
> {code}
> My code is in Scala but it works well with Beam 2.9.0.
> {code:java}
> val p = Pipeline.create(options)
> p.apply(s"${bu.name}_ReadFromKafka", KafkaIO.read()
> .withBootstrapServers(options.getBootstreapServers)
> .updateConsumerProperties(config)
> .withTopics(util.Arrays.asList(topicName))
> .withKeyDeserializer(classOf[LongDeserializer])
> .withValueDeserializer(classOf[StringDeserializer])
> .withConsumerFactoryFn(
> new KafkaTLSConsumerFactory(
> projectId, options.getSourceBucket, options.getTrustStoreGCSKey,
> options.getKeyStoreGCSKey)))
> .apply(s"${bu.name}_Convert", ParDo.of(new
> ConvertJSONTextToEPCTransaction(bu)))
> .apply(s"${bu.name}_WriteToBQ", BigQueryIO.write()
> .to(bqDestTable)
> .withSchema(schema)
> .withFormatFunction(new ConvertMessageToTable())
> .withCreateDisposition(BigQueryIO.Write.CreateDisposition.CREATE_NEVER)
> .withWriteDisposition(BigQueryIO.Write.WriteDisposition.WRITE_APPEND))
> }
> p.run
> {code}
> According to the error log, it failed at this part.
>
> [https://github.com/apache/beam/blob/v2.10.0/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/BoundedWindow.java#L81-L85]
> {code:java}
> private static Instant
> extractTimestampFromProto(RunnerApi.BeamConstants.Constants constant) {
> return new Instant(
> Long.parseLong(
>
> constant.getValueDescriptor().getOptions().getExtension(RunnerApi.beamConstant)));
> }
> {code}
> This constant come from this part.
>
> [https://github.com/apache/beam/blob/v2.10.0/sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/windowing/BoundedWindow.java#L48-L49]
> {code:java}
> public static final Instant TIMESTAMP_MIN_VALUE =
>
> extractTimestampFromProto(RunnerApi.BeamConstants.Constants.MIN_TIMESTAMP_MILLIS);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
