wwj6591812 commented on code in PR #7991:
URL: https://github.com/apache/paimon/pull/7991#discussion_r3321380158
##########
paimon-flink/paimon-flink-common/src/main/java/org/apache/paimon/flink/source/operator/ReadOperator.java:
##########
@@ -68,6 +69,7 @@ public class ReadOperator extends
AbstractStreamOperator<RowData>
private transient long emitEventTimeLag =
FileStoreSourceReaderMetrics.UNDEFINED;
private transient long idleStartTime = FileStoreSourceReaderMetrics.ACTIVE;
private transient Counter numRecordsIn;
Review Comment:
Thanks for the review and for raising this.
We would prefer to keep numRecordsIn. It is used for Flink I/O metrics, not
for limit enforcement anymore. In open(), we bind it to
getNumRecordsInCounter(), and in processElement() we increment it so emitted
rows are reflected in the standard numRecordsIn metric.
After this change, limit enforcement is handled only by RecordLimiter. The
old use of numRecordsIn in reachLimit() was part of the bug, so we moved that
responsibility out.
If we remove it, the existing numRecordsIn metric on this operator would no
longer be updated.
Thanks again for your review.
@JingsongLi
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
