[
https://issues.apache.org/jira/browse/BEAM-14365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thiago Nunes updated BEAM-14365:
--------------------------------
Description:
We got the following error when testing the Spanner change streams connector:
{code:java}
"java.lang.IllegalArgumentException: Expected size >= 0 but received
-36590.86666666666. at
org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory$DefaultGetSize.validateSize(ByteBuddyDoFnInvokerFactory.java:438)
at
org.apache.beam.sdk.io.gcp.spanner.changestreams.dofn.ReadChangeStreamPartitionDoFn$DoFnInvoker.invokeGetSize(Unknown
Source) at
org.apache.beam.fn.harness.FnApiDoFnRunner.calculateRestrictionSize(FnApiDoFnRunner.java:1182)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.trySplitForElementAndRestriction(FnApiDoFnRunner.java:1625)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.access$1800(FnApiDoFnRunner.java:142)
at
org.apache.beam.fn.harness.FnApiDoFnRunner$SplittableFnDataReceiver.trySplit(FnApiDoFnRunner.java:1113)
at
org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$SplittingMetricTrackingFnDataReceiver.trySplit(PCollectionConsumerRegistry.java:342)
at
org.apache.beam.fn.harness.BeamFnDataReadRunner.trySplit(BeamFnDataReadRunner.java:259)
at
org.apache.beam.fn.harness.control.ProcessBundleHandler.trySplit(ProcessBundleHandler.java:688)
at
org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:151)
at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:116)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833) " {code}
This indicates that the return value of getSize() in SDF is negative. Since the
fields in Progress cannot be negative, the negative number should come from the
local throughput estimator.
When this error occurs, the pipeline gets stuck and no progress is made. This
prevents users from enabling autoscaling when using the SpannerIO change
streams functionality.
was:
We got the following error when testing the Spanner change streams connector:
{code:java}
"java.lang.IllegalArgumentException: Expected size >= 0 but received
-36590.86666666666. at
org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory$DefaultGetSize.validateSize(ByteBuddyDoFnInvokerFactory.java:438)
at
org.apache.beam.sdk.io.gcp.spanner.changestreams.dofn.ReadChangeStreamPartitionDoFn$DoFnInvoker.invokeGetSize(Unknown
Source) at
org.apache.beam.fn.harness.FnApiDoFnRunner.calculateRestrictionSize(FnApiDoFnRunner.java:1182)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.trySplitForElementAndRestriction(FnApiDoFnRunner.java:1625)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.access$1800(FnApiDoFnRunner.java:142)
at
org.apache.beam.fn.harness.FnApiDoFnRunner$SplittableFnDataReceiver.trySplit(FnApiDoFnRunner.java:1113)
at
org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$SplittingMetricTrackingFnDataReceiver.trySplit(PCollectionConsumerRegistry.java:342)
at
org.apache.beam.fn.harness.BeamFnDataReadRunner.trySplit(BeamFnDataReadRunner.java:259)
at
org.apache.beam.fn.harness.control.ProcessBundleHandler.trySplit(ProcessBundleHandler.java:688)
at
org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:151)
at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:116)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833) " {code}
This indicates that the return value of getSize() in SDF is negative. Since the
fields in Progress cannot be negative, the negative number should come from the
local throughput estimator.
> [SpannerIO.readChangeStream] The reported backlog size should not be negative
> -----------------------------------------------------------------------------
>
> Key: BEAM-14365
> URL: https://issues.apache.org/jira/browse/BEAM-14365
> Project: Beam
> Issue Type: Bug
> Components: io-java-gcp
> Affects Versions: 2.39.0
> Reporter: Hengfeng Li
> Priority: P2
> Fix For: 2.39.0
>
>
> We got the following error when testing the Spanner change streams connector:
> {code:java}
> "java.lang.IllegalArgumentException: Expected size >= 0 but received
> -36590.86666666666. at
> org.apache.beam.sdk.transforms.reflect.ByteBuddyDoFnInvokerFactory$DefaultGetSize.validateSize(ByteBuddyDoFnInvokerFactory.java:438)
> at
> org.apache.beam.sdk.io.gcp.spanner.changestreams.dofn.ReadChangeStreamPartitionDoFn$DoFnInvoker.invokeGetSize(Unknown
> Source) at
> org.apache.beam.fn.harness.FnApiDoFnRunner.calculateRestrictionSize(FnApiDoFnRunner.java:1182)
> at
> org.apache.beam.fn.harness.FnApiDoFnRunner.trySplitForElementAndRestriction(FnApiDoFnRunner.java:1625)
> at
> org.apache.beam.fn.harness.FnApiDoFnRunner.access$1800(FnApiDoFnRunner.java:142)
> at
> org.apache.beam.fn.harness.FnApiDoFnRunner$SplittableFnDataReceiver.trySplit(FnApiDoFnRunner.java:1113)
> at
> org.apache.beam.fn.harness.data.PCollectionConsumerRegistry$SplittingMetricTrackingFnDataReceiver.trySplit(PCollectionConsumerRegistry.java:342)
> at
> org.apache.beam.fn.harness.BeamFnDataReadRunner.trySplit(BeamFnDataReadRunner.java:259)
> at
> org.apache.beam.fn.harness.control.ProcessBundleHandler.trySplit(ProcessBundleHandler.java:688)
> at
> org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:151)
> at
> org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:116)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
> at
> java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
> at java.base/java.lang.Thread.run(Thread.java:833) " {code}
> This indicates that the return value of getSize() in SDF is negative. Since
> the fields in Progress cannot be negative, the negative number should come
> from the local throughput estimator.
>
> When this error occurs, the pipeline gets stuck and no progress is made. This
> prevents users from enabling autoscaling when using the SpannerIO change
> streams functionality.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
