wwj6591812 commented on code in PR #8116:
URL: https://github.com/apache/paimon/pull/8116#discussion_r3463938582
##########
paimon-core/src/main/java/org/apache/paimon/table/source/KeyValueTableRead.java:
##########
@@ -129,11 +141,97 @@ public InnerTableRead withTopN(TopN topN) {
@Override
public InnerTableRead withLimit(int limit) {
- initialized().forEach(r -> r.withLimit(limit));
this.limit = limit;
+ refreshReaderConfig();
return this;
}
+ @Override
+ public RecordReader<InternalRow> createReader(List<Split> splits) throws
IOException {
+ if (limit == null || limit <= 0) {
+ return createConcatenatedReader(splits);
+ }
+
+ if (needsTableLevelLimit(splits)) {
+ return createReaderWithGlobalLimit(splits);
+ }
+ return createConcatenatedReader(splits);
+ }
+
+ private boolean needsTableLevelLimit(List<Split> splits) {
+ if (splits.size() > 1) {
+ return true;
+ }
+ if (!isMergeReadLimitActive(splits.get(0))) {
Review Comment:
Thanks for the follow-up. @JingsongLi
You're right — when filters or an empty table produce no splits,
needsTableLevelLimit() could call splits.get(0) and throw
IndexOutOfBoundsException.
I fixed this by short-circuiting in createReader() when splits is empty
(returning an empty reader directly) and adding the same guard in
needsTableLevelLimit(). I also added testReadWithLimitOnEmptyPlan as a
regression test for withLimit over an empty plan.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
