ktalluri456 opened a new pull request, #36737: URL: https://github.com/apache/beam/pull/36737
Fixes #36736 ### What is the purpose of this change? This pull request introduces state sampling for timer processing within the Python SDK. This adds a new metric, `process-timers-millis`, which allows runners to more accurately track the time spent on timer processing. This is particularly valuable for streaming pipelines with high timer backlogs, as it provides a key signal for autoscaling algorithms to make more informed decisions. ### The key changes include: * The `scoped_state` method in `statesampler.py` has been updated to accept a `suffix` argument, allowing for more flexible counter names. * The `process_timer` method in `operations.py` now wraps the timer processing logic in a `scoped_state` to measure the execution time. * A new unit test, `test_timer_sampler`, has been added to `statesampler_test.py` to verify the new counter. --- **Follow this checklist to help us incorporate your contribution quickly and easily:** - [X] Mention the appropriate issue in your description (see `Fixes #...` above). - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
