xuyangzhong commented on code in PR #1934:
URL: https://github.com/apache/fluss/pull/1934#discussion_r2494349801
##########
fluss-flink/fluss-flink-common/src/main/java/org/apache/fluss/flink/source/FlinkTableSource.java:
##########
@@ -572,7 +576,11 @@ && hasPrimaryKey()
}
}
}
- return Result.of(acceptedFilters, remainingFilters);
+
+ // We cannot determine whether this source will ultimately be used
as a scan source or
+ // a lookup source. Since fluss lookup sources cannot accept
filters yet, to be safe, we
+ // return all filters to the Flink planner.
+ return Result.of(acceptedFilters, filters);
Review Comment:
Currently, there is no way to determine whether the source during filter
pushdown is a scan source or a lookup source in Flink. Given the current
situation, in terms of performance and correctness, I believe we should always
prioritize correctness.
Another approach, which might not be user-friendly, is to explicitly raise
an error in `getLookupRuntimeProvider` when we detect that `singleRowFilter` or
`partitionFilters` is not null (indicating that some filters have been
applied). This error would inform users that lookup tables do not support
filter pushdown and suggest adding a specific table options (e.g., enable
filter push down = false) via a hint on the lookup table. However, this
approach of raising errors could potentially harm the user experience. WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
