Abacn opened a new pull request, #40911: URL: https://github.com/apache/beam/pull/40911
`WriteImpl._expand_bounded` previously applied `WindowInto(GlobalWindows())` unconditionally to all input records before writing bundles, even when the input `PCollection` was already using the default global windowing strategy. In typical batch pipelines, this added an unnecessary per-element `WindowInto` pass across the entire dataset (~22% overhead in bounded write benchmarks). Check `windowing.is_default()` at pipeline expansion time and only apply `WindowInto(GlobalWindows())` when the input `PCollection` is not already in the default global window. ======== Benchmark Results (source: https://gist.github.com/Abacn/93fa3168450bf36838e70d853a77d2e3) | Benchmark | Baseline Median (sec / elem) | This change (sec / elem) | Speedup | Time Reduction | | :--- | ---: | ---: | ---: | ---: | | `WriteImplBoundedPipeline` | `1.643e-05` | `1.281e-05` | **1.28x** | **-22.0%** | <!-- Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). - [ ] Replace these instructions with a description of your change and why you have made it. - [ ] Mention the appropriate issue in your description (for example: `addresses #123` or `fixes #456`). - [ ] Update `CHANGES.md` with noteworthy changes. See [CONTRIBUTING.md]() for more details. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
