JingsongLi commented on code in PR #8116:
URL: https://github.com/apache/paimon/pull/8116#discussion_r3503268601
##########
paimon-core/src/main/java/org/apache/paimon/table/source/KeyValueTableRead.java:
##########
@@ -134,6 +136,31 @@ public InnerTableRead withLimit(int limit) {
return this;
}
+ @Override
+ public RecordReader<InternalRow> createReader(List<Split> splits) throws
IOException {
+ return LimitRecordReader.limit(super.createReader(splits), limit);
+ }
+
+ @Override
+ public RecordReader<InternalRow> createReader(Split split) throws
IOException {
+ // Query-auth filters run after split read; skip merge-read limit for
those splits.
+ if (!hasLateAppliedRowFilter(split)) {
+ return super.createReader(split);
+ }
+
+ Integer savedLimit = this.limit;
+ this.limit = null;
+ initialized().forEach(r -> r.withLimit(null));
+ try {
+ return super.createReader(split);
Review Comment:
For the single-split API this drops the limit completely when query auth has
a row filter. The method clears `this.limit` to avoid applying it before
`authResult.doAuth(...)`, but then returns `super.createReader(split)`
directly, so `table.newRead().withLimit(10).createReader(queryAuthSplit)` can
read all authorized rows. The new tests cover `createReader(List<Split>)`,
whose outer `LimitRecordReader` restores the total limit, but callers and
source readers also use the public `createReader(Split)` path. Could we wrap
this returned reader with `LimitRecordReader.limit(..., savedLimit)` after auth
filtering, so the limit is still enforced after the late row filter?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
