JingsongLi commented on code in PR #10959:
URL: https://github.com/apache/paimon/pull/10959#discussion_r4451387944
##########
paimon-core/src/main/java/org/apache/paimon/operation/RawFileSplitRead.java:
##########
@@ -146,6 +148,10 @@ public SplitRead<InternalRow> withReadType(RowType
readRowType) {
public RawFileSplitRead withFilter(Predicate predicate) {
if (predicate != null) {
this.filters = splitAnd(predicate);
+ if (rowTrackingEnabled) {
+ // Row ids and sequence numbers may only be in the file meta,
not in the file.
+ this.filters = excludePredicateWithFields(filters,
ROW_TRACKING_FIELD_NAMES);
Review Comment:
[P2] Preserve residual-filter state before applying an early limit
Removing the only predicate makes `filters` empty, so `createReader` now
applies both its format limit and `LimitRecordReader` before
`AbstractDataTableRead` executes the original row-tracking predicate. On this
head, adding `.withLimit(1)` to the new test's
`table.newReadBuilder().withFilter(filter)` is enough to make its first
assertion fail: `_ROW_ID = 2` returns `[]` instead of the existing `3, 30, 2,
2` row. The table contains three rows across two commits, and the early cap
consumes a nonmatching first row. The scan-side nonpartition-filter guard does
not prevent this reader-level cap. Please preserve knowledge of the deferred
predicate and avoid early format/reader limits (or apply the limit after the
residual), with coverage for both metadata fields combined with LIMIT.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]