Abacn commented on issue #18592: URL: https://github.com/apache/beam/issues/18592#issuecomment-3427017519
> One previously identified issue was that teardown may not be invoked on a DoFn if there is an exception calling setup on a different fused DoFn. Yes, link here: https://github.com/apache/beam/pull/32522/files#r1777301355 > However I believe that if the dataflow runner abandons a MapTaskExecutor for some reason not related to an exception in user processing (cache timeout or exception during backend commit) that we may not be invoking the teardown method. This can lead to leaking resources if they are created in setup and require teardown to close them. An example of this is the KafkaProducer in KafkaWriter [here](sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaWriter.java#L57). The link not working, pasted a perma-link here for reference: https://github.com/apache/beam/blob/c22665c5111709a9181cf5a620339326ba97e88b/sdks/java/io/kafka/src/main/java/org/apache/beam/sdk/io/kafka/KafkaWriter.java#L57 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
