lofifnc commented on PR #39310: URL: https://github.com/apache/beam/pull/39310#issuecomment-5000569431
> Thanks, since #39309 mentioned benchmark numbers. Have you verified that this fixes the performance issue? To run a pipeline against modified Beam code, one can refer to this doc: > > https://github.com/apache/beam/blob/master/contributor-docs/code-change-guide.md#run-your-pipeline-with-modified-beam-code Thanks for looking into this! We tested this on a production-scale Dataflow streaming workload. The test used 55 fixed n4-highmem-8 workers with 16 worker-harness threads, comparing Beam 2.72.0 plus the current stock Beam 2.72, patched with this PR, production job over the same time window. The PR build averaged 49.29% worker CPU at 141,802 records/s. Stock Beam 2.72 averaged 72.77% CPU at 141,808 records/s. In the same timeframe. This is a 23.48 percentage-point, or 32.27% relative, CPU reduction at effectively identical throughput. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
