RustedBones opened a new pull request, #30453:
URL: https://github.com/apache/beam/pull/30453
We've noticed that a failure to process a bigquery `Mutation` (for instance
due to a coder exception), results with the following error in the logs:
```
Batcher was not closed properly!!! Make sure to call close().
java.lang.RuntimeException: Batcher allocation site
at
com.google.api.gax.batching.BatcherImpl$BatcherReference.<init>(BatcherImpl.java:552)
at com.google.api.gax.batching.BatcherImpl.<init>(BatcherImpl.java:210)
at
com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub.newMutateRowsBatcher(EnhancedBigtableStub.java:690)
at
com.google.cloud.bigtable.data.v2.BigtableDataClient.newBulkMutationBatcher(BigtableDataClient.java:1113)
at
com.google.cloud.bigtable.data.v2.BigtableDataClient.newBulkMutationBatcher(BigtableDataClient.java:1080)
at
org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceImpl$BigtableWriterImpl.<init>(BigtableServiceImpl.java:501)
at
org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceImpl.openForWriting(BigtableServiceImpl.java:131)
at
org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceImpl.openForWriting(BigtableServiceImpl.java:93)
at
org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$BigtableWriterFn.startBundle(BigtableIO.java:1248)
at
org.apache.beam.sdk.io.gcp.bigtable.BigtableIO$BigtableWriterFn$DoFnInvoker.invokeStartBundle(Unknown
Source)
at
org.apache.beam.fn.harness.FnApiDoFnRunner.startBundle(FnApiDoFnRunner.java:797)
at
org.apache.beam.fn.harness.data.PTransformFunctionRegistry.lambda$register$0(PTransformFunctionRegistry.java:116)
at
org.apache.beam.fn.harness.control.ProcessBundleHandler.processBundle(ProcessBundleHandler.java:530)
at
org.apache.beam.fn.harness.control.BeamFnControlClient.delegateOnInstructionRequestType(BeamFnControlClient.java:150)
at
org.apache.beam.fn.harness.control.BeamFnControlClient$InboundObserver.lambda$onNext$0(BeamFnControlClient.java:115)
at
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at
org.apache.beam.sdk.util.UnboundedScheduledExecutorService$ScheduledFutureTask.run(UnboundedScheduledExecutorService.java:163)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
```
The `bigtableWriter` and `serviceEntry` resources are allocated in the
`StartBundle` and only closed in the `FinishBundle`. If an exception is raised
during the `ProcessElement`, `FinishBunlde` is not called, resulting in a
memory leak when retrying the step.
This change makes sure to close those resources during the `TearDown` phase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
