prodriguezdefino opened a new issue, #22416: URL: https://github.com/apache/beam/issues/22416
### What happened? After the inclusion of the propagation of successful inserts in v2.37.0 pipelines that were able to handle very large throughputs (+10GB/s) started to see problems when recovering from backlog or when delays happened from upstream and there were already at max num of workers. It seems that holding the TupleTag data after insertion generates some pressure on the workers that slowdown the pipeline reducing the max throughput at a given number of user workers. Introducing an option to enable/disable this propagation (enabled by default for backwards compatibility) should help to recover part of that lost throughput on large volume pipelines. ### Issue Priority Priority: 1 ### Issue Component Component: io-java-gcp -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
