QuakeWang commented on code in PR #224:
URL: https://github.com/apache/paimon-rust/pull/224#discussion_r3050301043
##########
crates/integrations/datafusion/src/filter_pushdown.rs:
##########
@@ -58,6 +58,22 @@ pub(crate) fn build_pushed_predicate(filters: &[Expr],
fields: &[DataField]) ->
}
}
+/// Whether it is safe to pass a row-count hint down to paimon-core planning.
+///
+/// This stays intentionally narrow: the hint is only safe when there are no
+/// filters, or when every filter is exact at the table-provider boundary.
+pub(crate) fn can_push_down_limit_hint(
+ filters: &[Expr],
+ fields: &[DataField],
+ partition_keys: &[String],
+) -> bool {
+ filters.is_empty()
+ || filters.iter().all(|filter| {
+ classify_filter_pushdown(filter, fields, partition_keys)
Review Comment:
I moved the scan-owned part of this judgment into paimon-core. The remaining
DataFusion-side check is only for residual filters that are not visible to
`ReadBuilder::with_filter(...)`, because `with_filter` only receives the
translated Paimon `Predicate`.
Unsupported or inexact DataFusion filters may still remain above the scan,
and that residual part is still needed to decide whether `LIMIT` can be pushed
safely. So I kept only this residual-filter guard in the DataFusion connector.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
