navina commented on code in PR #9621:
URL: https://github.com/apache/pinot/pull/9621#discussion_r1000144296
##########
pinot-spi/src/main/java/org/apache/pinot/spi/stream/PartitionLagState.java:
##########
@@ -27,12 +27,20 @@ public class PartitionLagState {
protected final static String NOT_CALCULATED = "NOT_CALCULATED";
/**
- * Defines how far away the current record's offset / pointer is from
upstream latest record
+ * Defines how far behind the current record's offset / pointer is from
upstream latest record
* The distance is based on actual record count.
*/
public String getRecordsLag() {
return NOT_CALCULATED;
}
- // TODO: Define record availability lag ($latest_record_consumption_time -
$latest_record_ingestion_time)
+ /**
+ * Defines how soon after record ingestion was the record consumed by Pinot.
That is, the difference between the
+ * time the record was consumed and the time at which the record was
ingested upstream.
+ *
+ * @return Lag value in milliseconds
+ */
+ public String getRecordAvailabilityLag() {
Review Comment:
I didn't want to call it "offset" lag as that term doesn't makes sense for
anything other than kafka. in fact, `recordsLag` can be computed only for
Kafka. Hence, I am using the same term as kafka's metric name for lag.
For every other system, we can only compute a time-based lag. and it
indicates when the record becomes available for consumption. I was considering
calling it `getRecordAvailabilityLagMs`, if that helps clarify the meaning?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
