jugomezv commented on code in PR #9977:
URL: https://github.com/apache/pinot/pull/9977#discussion_r1051009453
##########
pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java:
##########
@@ -45,7 +45,10 @@ public enum ServerGauge implements AbstractMetrics.Gauge {
// Dedup metrics
DEDUP_PRIMARY_KEYS_COUNT("dedupPrimaryKeysCount", false),
CONSUMPTION_QUOTA_UTILIZATION("ratio", false),
- JVM_HEAP_USED_BYTES("bytes", true);
+ JVM_HEAP_USED_BYTES("bytes", true),
+ // Consumer lag metrics
+ LLC_RECORDS_LAG("recordCount", false),
+ LLC_AVAILABILITY_LAG_MS("availabilityLagMs", false);
Review Comment:
Our argument to not have a per partition metric is that it just clouds the
user view, what we care about in most cases is the worst case scenario which is
what we are handling. Our SREs complained about the per metrica approach as it
would make their job harder, they wanted a single point metric that tells them
the worst case scenario.
Ack on this having removed the support for offset, but I think that was
important.
As for aging, imagine you collect one sample that says event delay last
measure was 100ms and then we do get stuck committing a segment, that may take
several seconds so the best estimate of consumption delay is the age of the
last worst sample plus the sample itself. If we keep posting the sample we are
giving the wrong idea to the customer. When we do try to consume and no events
are found we display 0 delay as if there were events we will consume them with
no delay. Please let me know if the meeting I set for next week suits your
schedule and if not let me know what time will be best.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
