adriangb commented on code in PR #15301:
URL: https://github.com/apache/datafusion/pull/15301#discussion_r2020151449
##########
datafusion/physical-plan/src/sorts/sort.rs:
##########
@@ -1224,6 +1245,28 @@ impl ExecutionPlan for SortExec {
.with_preserve_partitioning(self.preserve_partitioning()),
)))
}
+
+ // Pass though filter pushdown.
+ // This often happens in partitioned plans with a TopK because we end up
with 1 TopK per partition + a final TopK at the end.
+ // Implementing this pass-through allows global/top/final TopK to push
down filters to the partitions.
+ fn push_down_filter(
+ &self,
+ expr: Arc<dyn PhysicalExpr>,
+ ) -> Result<Option<Arc<dyn ExecutionPlan>>> {
Review Comment:
This won't do anything for the multi-partition case _until_ we implement
filter pushdown for `SortingMergeExec`. At that point this method becomes
relevant because it forwards the filters from the `SortingMergeExec` down. We
_could_ do something here where we simplify / merge the bounds, but (1) that
could get messy and (2) it's an internal implementation detail we can easily
tweak in the future.
But also maybe the right thing to do is for the final consumers of these
filters to do a final optimize pass over the snapshotted physicalexprs and that
does the merging?
Either way... future work.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
