neilconway commented on PR #22343: URL: https://github.com/apache/datafusion/pull/22343#issuecomment-4593920173
@alamb To me, this falls into the category of stuff that might break when you upgrade performance-sensitive apps that are built on a system with a declarative query language / query optimizer. I think it's untenable to promise that _no_ user workloads will see performance regressions from new versions. The reordering done by this PR is intentionally very conservative / simple, so I would be surprised if we see widespread issues in the field arising from this change. If a user's workload is *that* sensitive to the exact predicate evaluation order, they might be better off encoding their filtering criteria as a custom UDF. Dynamic filter reordering would probably help most cases in practice (albeit it might make the actual runtime behavior _more_ unpredictable). At some point in the future, we could also potentially ship either a cost-based optimizer (where users could annotate individual UDFs with cost estimates), and/or some facility for manually specifying properties of the evaluation order (e.g., "hints"). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
