Dharshika-11 opened a new pull request, #39767: URL: https://github.com/apache/beam/pull/39767
Summary This PR updates PGBKCVOperation to bound its precombine table based on estimated in-memory size in addition to the existing key-count limit. Changes Added an in-memory size limit using max_bytes. PGBKCVOperation now tracks estimated memory usage through estimated_bytes and get_deep_size. Added default memory limits: 10 MB for general accumulators. 100 MB for tiny accumulators such as Count, Mean, sum, min, and max. The table now flushes when the estimated memory usage exceeds the configured limit, helping prevent OOM situations caused by very large accumulators. Updated operations.pxd with the new Cython declarations: max_bytes estimated_bytes _is_tiny_accumulator Added unit tests covering: Memory-limit-based flushing. Existing key-count-based flushing. Custom max_bytes and max_keys parameters. Correct output after flushing. Issue Fixes #39754 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
