wwj6591812 commented on code in PR #8116:
URL: https://github.com/apache/paimon/pull/8116#discussion_r3522835534
##########
paimon-core/src/main/java/org/apache/paimon/table/source/KeyValueTableRead.java:
##########
@@ -134,6 +136,31 @@ public InnerTableRead withLimit(int limit) {
return this;
}
+ @Override
+ public RecordReader<InternalRow> createReader(List<Split> splits) throws
IOException {
+ return LimitRecordReader.limit(super.createReader(splits), limit);
+ }
+
+ @Override
+ public RecordReader<InternalRow> createReader(Split split) throws
IOException {
+ // Query-auth filters run after split read; skip merge-read limit for
those splits.
+ if (!hasLateAppliedRowFilter(split)) {
+ return super.createReader(split);
Review Comment:
Thanks for the review, @JingsongLi.
You're right — the single-split createReader(Split) path only applied the
table-level LimitRecordReader fallback for QueryAuthSplit, so when merge-read
limit was disabled for other reasons (e.g. non-PK filter, forceKeepDelete,
partial-update/aggregation, or deletion vectors), the public single-split API
could return all matching rows with no limit enforced. I've generalized the
routing via MergeFileSplitRead#isMergeReadLimitActive() so any split whose
lower-level read cannot apply merge-read limit uses the table-level fallback,
and added
testReadWithLimitThroughSingleSplitCreateReaderWithNonPrimaryKeyFilter to cover
the public single-split API. Please take another look when you have a chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
