navina commented on code in PR #9977:
URL: https://github.com/apache/pinot/pull/9977#discussion_r1049012304
##########
pinot-common/src/main/java/org/apache/pinot/common/metrics/ServerGauge.java:
##########
@@ -45,7 +45,10 @@ public enum ServerGauge implements AbstractMetrics.Gauge {
// Dedup metrics
DEDUP_PRIMARY_KEYS_COUNT("dedupPrimaryKeysCount", false),
CONSUMPTION_QUOTA_UTILIZATION("ratio", false),
- JVM_HEAP_USED_BYTES("bytes", true);
+ JVM_HEAP_USED_BYTES("bytes", true),
+ // Consumer lag metrics
+ LLC_RECORDS_LAG("recordCount", false),
+ LLC_AVAILABILITY_LAG_MS("availabilityLagMs", false);
Review Comment:
>show how far behind our consumer is from the input stream irrespective of
what the engine is doing
my bad. I corrected my typo above. It is how far our consumer is with
respect to the last processed message.
> The idea is to know how stale the data is when queries are being served in
a busy production environment.
The idea is simply to track the progress the consumer is making regardless
of what the engine is doing. So, yes, it will increase when we are not polling
the stream. that is expected and that is what we want to know.
> Does this take care of the lag when a server is building segments or in
HOLD state awaiting segment completion?
Can you explain what do you mean by this? availability lag should increase
when the server is not polling and is busy with segment building/committing.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
