Hengfeng Li created BEAM-14365:
----------------------------------
Summary: [SpannerIO.readChangeStream] The reported backlog size
should not be negative
Key: BEAM-14365
URL: https://issues.apache.org/jira/browse/BEAM-14365
Project: Beam
Issue Type: Bug
Components: io-java-gcp
Affects Versions: 2.39.0
Reporter: Hengfeng Li
Fix For: 2.39.0
We got the following error when testing the Spanner change streams connector:
```
"java.lang.IllegalArgumentException: Expected size >= 0 but received
-36590.86666666666. at
org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory$DefaultGetSize.validateSize(ByteBuddyDoFnInvokerFactory.java:438)
at
org.apache.beam.sdk.io.gcp.spanner.changestreams.dofn.ReadChangeStreamPartitionDoFn$DoFnInvoker.invokeGetSize(Unknown
Source) at
org.apache.beam.fn.harness.FnApiDoFnRunner.calculateRestrictionSize(FnApiDoFnRunner.java:1182)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.trySplitForElementAndRestriction(FnApiDoFnRunner.java:1625)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.access$1800(FnApiDoFnRunner.java:142)
at
org.apache.beam.fn.harness.FnApiDoFnRunner$SplittableFnDataReceiver.trySplit(FnApiDoFnRunner.java:1113)
at
org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$SplittingMetricTrackingFnDataReceiver.trySplit(PCollectionConsumerRegistry.java:342)
at
org.apache.beam.fn.harness.BeamFnDataReadRunner.trySplit(BeamFnDataReadRunner.java:259)
at
org.apache.beam.fn.harness.control.ProcessBundleHandler.trySplit(ProcessBundleHandler.java:688)
at
org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:151)
at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:116)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833) "
```
This indicates that the return value of getSize() in SDF is negative. Since the
fields in Progress cannot be negative, the negative number should come from the
local throughput estimator.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
