chenjian2664 commented on code in PR #10959:
URL: https://github.com/apache/paimon/pull/10959#discussion_r4451474999
##########
paimon-core/src/main/java/org/apache/paimon/operation/RawFileSplitRead.java:
##########
@@ -146,6 +148,10 @@ public SplitRead<InternalRow> withReadType(RowType
readRowType) {
public RawFileSplitRead withFilter(Predicate predicate) {
if (predicate != null) {
this.filters = splitAnd(predicate);
+ if (rowTrackingEnabled) {
+ // Row ids and sequence numbers may only be in the file meta,
not in the file.
+ this.filters = excludePredicateWithFields(filters,
ROW_TRACKING_FIELD_NAMES);
Review Comment:
Good catch, thanks. `RawFileSplitRead` now remembers when a row tracking
conjunct was dropped from the pushed filters. In that case it builds the format
readers without TopN or limit and skips `LimitRecordReader`, so the remaining
predicate runs on the rows before anything caps them. The scan side was already
safe: `hasNonPartitionFilter` is set from the original predicate, so the
scan-level limit and TopN are not pushed. The test now covers `_ROW_ID` and
`_SEQUENCE_NUMBER`, each with and without `LIMIT 1`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]