xiangfu0 opened a new pull request, #16745: URL: https://github.com/apache/pinot/pull/16745
This pull request adds support for collecting Kafka consumer metrics using a scope-based approach in both the `pinot.yml` and `server.yml` JMX Prometheus Java Agent configuration files. The new rules enable more granular monitoring of Kafka consumer activity by capturing metrics at the consumer, topic, and partition levels, with appropriate labeling for easier identification and aggregation. **Kafka Consumer Metrics Collection:** * Added rules to capture consumer-level metrics (e.g., fetch rate, latency, bytes consumed) with labels for `table`, `topic_name`, `partition`, and `scope=consumer` in both `pinot.yml` and `server.yml`. [[1]](diffhunk://#diff-7f4df59f77472c0840d9b41bf3ec680ed286cb34c504079f798a5dac652d6919R599-R626) [[2]](diffhunk://#diff-2eab1bf3447d4d9c61dfc80e6a0299e3785045f368019d2d9d593f95515cf39eR88-R115) * Added rules for topic-level metrics (e.g., bytes consumed rate, fetch size) with labels for `table`, `topic_name`, `partition`, and `scope=topic` in both configuration files. [[1]](diffhunk://#diff-7f4df59f77472c0840d9b41bf3ec680ed286cb34c504079f798a5dac652d6919R599-R626) [[2]](diffhunk://#diff-2eab1bf3447d4d9c61dfc80e6a0299e3785045f368019d2d9d593f95515cf39eR88-R115) * Added rules for partition-level metrics (e.g., records lag/lead) with labels for `table`, `topic_name`, `partition`, and `scope=partition` in both configuration files. [[1]](diffhunk://#diff-7f4df59f77472c0840d9b41bf3ec680ed286cb34c504079f798a5dac652d6919R599-R626) [[2]](diffhunk://#diff-2eab1bf3447d4d9c61dfc80e6a0299e3785045f368019d2d9d593f95515cf39eR88-R115) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
