prashantwason opened a new pull request, #18085: URL: https://github.com/apache/hudi/pull/18085
### Describe the issue this Pull Request addresses Add metrics capabilities to delta streamer for better observability: 1. Emit success/failure metrics when delta streamer job is finished 2. Emit the number of commits to process in the delta streamer job 3. Emit the number of unprocessed commits in the source table timeline ### Summary and Changelog This PR adds new metrics to the HoodieStreamer for improved monitoring and observability: - Added `emitStreamerJobSuccessMetrics()` method to emit a success gauge when job completes successfully - Added `emitStreamerJobFailedMetrics()` method to emit a failure gauge when job fails - Added `updateHoodieIncrSourceMetrics(long commitsInProgress, long unprocessedCommits)` method to track commits in progress and unprocessed commits - Added `reportFailureMetrics()` method to StreamSync for reporting failure metrics - Updated `SqlFileBasedSource` and `SqlSource` to accept `HoodieIngestionMetrics` parameter - Updated corresponding test files ### Impact No public API changes. This adds internal metrics that can be consumed by monitoring systems. ### Risk Level low - This change only adds new metrics without modifying existing behavior. ### Documentation Update none - These are internal metrics additions that don't require documentation updates. ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
