[
https://issues.apache.org/jira/browse/BEAM-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16686622#comment-16686622
]
Wout Scheepers commented on BEAM-4796:
--------------------------------------
I'm not using any windowing at all.
However, I didn't notice the following exception:
{code:java}
"java.lang.NoSuchMethodError:
org.apache.beam.model.pipeline.v1.RunnerApi$BeamConstants$Constants.getValueDescriptor()Lorg/apache/beam/vendor/grpc/v1_13_1/com/google/protobuf/Descriptors$EnumValueDescriptor;
at
org.apache.beam.sdk.transforms.windowing.BoundedWindow.extractTimestampFromProto(BoundedWindow.java:84)
at
org.apache.beam.sdk.transforms.windowing.BoundedWindow.<clinit>(BoundedWindow.java:49)
at
org.apache.beam.runners.dataflow.worker.WindmillTimeUtils.windmillToHarnessTimestamp(WindmillTimeUtils.java:49)
at
org.apache.beam.runners.dataflow.worker.WindmillTimeUtils.windmillToHarnessWatermark(WindmillTimeUtils.java:34)
at
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.dispatchLoop(StreamingDataflowWorker.java:902)
at
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker.access$200(StreamingDataflowWorker.java:143)
at
org.apache.beam.runners.dataflow.worker.StreamingDataflowWorker$1.run(StreamingDataflowWorker.java:588)
at java.lang.Thread.run(Thread.java:745) "
{code}
It seems that my problem might be unrelated to spanner but to a protobuf or
grpc dependency. I checked, and I am using the right ones:
protobuf-java com.google.protobuf 3.6.0
grpc-all io.grpc 1.13.1
Any ideas?
> SpannerIO waits for all input before writing
> --------------------------------------------
>
> Key: BEAM-4796
> URL: https://issues.apache.org/jira/browse/BEAM-4796
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.5.0, 2.6.0, 2.7.0, 2.8.0
> Reporter: Niel Markwick
> Assignee: Niel Markwick
> Priority: Major
> Fix For: 2.9.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> SpannerIO.Write waits for all input in the window to arrive before getting
> the schema:
> [https://github.com/apache/beam/blame/release-2.5.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/spanner/SpannerIO.java#L841]
>
> In streaming mode, this is not an issue, but in batch mode, this causes the
> pipeline to stall until all input is read, which could be a significant
> amount of time (and temp data).
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
