navina commented on code in PR #9977:
URL: https://github.com/apache/pinot/pull/9977#discussion_r1050954430
##########
pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java:
##########
@@ -45,7 +45,10 @@ public enum ServerGauge implements AbstractMetrics.Gauge {
// Dedup metrics
DEDUP_PRIMARY_KEYS_COUNT("dedupPrimaryKeysCount", false),
CONSUMPTION_QUOTA_UTILIZATION("ratio", false),
- JVM_HEAP_USED_BYTES("bytes", true);
+ JVM_HEAP_USED_BYTES("bytes", true),
+ // Consumer lag metrics
+ LLC_RECORDS_LAG("recordCount", false),
+ LLC_AVAILABILITY_LAG_MS("availabilityLagMs", false);
Review Comment:
@jugomezv :
> explosion of metrics if we do one metric for partition.
can you clarify why adding this partition-level metric would cause metric
explosion when the realtime engine already has a bunch of per-partition-level
metric (eg. `HIGHEST_STREAM_OFFSET_CONSUMED` , `LLC_PARTITION_CONSUMING`,
`UPSERT_PRIMARY_KEYS_COUNT` etc )?
> These extra items do make sense for time based but not for offset based so
I do think we should keep offset and time based lag separate. May be focus this
pr on offset and the other one in lag.
This PR does not do offset-based lag as it is pretty expensive. The initial
version of the PR did include the offset lag. but there was a flag that made
sure it doesn't actually emit the metric. I have updated the PR since to only
publish time-based lag.
> We also address issues like aging of time lag measures and handling of
events that affect partition hosting.
By "aging of time lag measure", do you mean incrementing the lag when we are
not polling for messages? That would only make the lag trend-up when: 1. there
is no data in the input stream or 2. consumption rate is lower than the
ingestion rate . From a monitoring stand-point, it is hard to define a rule
that would not result in false positives.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
