adriangb commented on PR #15566: URL: https://github.com/apache/datafusion/pull/15566#issuecomment-2791076569
> Maybe it is possible to move the recursion into the optimizer rule but still keep a `ExecutionPlan` method by making a complex call signature, maybe something like this: > I think my original suggestions were around trying to simplify the changes to ExecutionPlan trait. As I recall the original draft PR had at least three new methods on `ExecutionPlan` that had to be kept in sync: > > 1. Could the node itself handle any filters internally? > 2. Could the node push filters past itself (the equivalent of what `try_swap_with_projection` does)? > 3. Did the node produce any dynamic filters? > > Maybe it is possible to move the recursion into the optimizer rule but still keep a `ExecutionPlan` method by making a complex call signature, maybe something like this: Yes precisely. I think we can collapse (1) & (2) at the cost of a more complex API for the collapsed method. But I think we'll always need 2 methods because of how the recursion works: the current node can't know what "remainder" filters it has the option to handle until we've pushed down into its children, but to push down into its children we first need to ask it which filters it has / allows to be pushed down to each child. I can try to rework to go back to something like that, but I expect the APIs to be much more complex for _all_ cases (not just for the actually complicated ones) and the diff is going to be much larger. Are there concrete things that the current approach falls short on that moving the recursion into the OptimizerRule would solve? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
