alamb commented on PR #22343: URL: https://github.com/apache/datafusion/pull/22343#issuecomment-4593629813
> My broader concern is that it seems pretty fragile to guarantee that filter evaluation order matches query text order. I could see this very easily being violated by other optimizations, either present or future. For example, what about predicate pushdown? If the query has a list of predicates and some of them can be pushed down while others cannot, that would result in predicate evaluation order becoming inconsistent with the query text order. 🤔 that is a good point > **tldr** -- If we actually want to guarantee that evaluation order matches query text order, considerably more work is required than just gating this PR in a boolean variable. Do we have any signal that this is a real pain point with users? @neilconway you make great points. The scenario I am imagining in my head is 1. Someone upgrades to the version of DataFusion that has this feature 2. Their query performance gets worse because filter order is different due to this change 3. There is no way to work around the issue without changing the code You have an excellent point that we don't have any real way to prevent this scenario from happening with other optimizer passes / changes now other than hoping we prevent it with code review 🤔 How do you think we should proceed? Heuristic reordering and then rely on stuff like @adriangb dynamic reordering to avoid the regression? - https://github.com/apache/datafusion/pull/22698 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
