alamb commented on issue #8227: URL: https://github.com/apache/datafusion/issues/8227#issuecomment-2698552710
@clflushopt I don't really know as I am not driving the statistics rework myself and thus don't have much visibility into what is planned The only remaining thing I know of that comes to mind is - https://github.com/apache/datafusion/issues/8229 but that is more of an API exercise rather than algorithms. Another thing that comes to mind is to look at / extend the existing https://github.com/apache/datafusion/blob/main/datafusion/physical-expr/src/intervals/cp_solver.rs code / show how it is useful I was playing around with duckdb this morning. Somehow it uses its constrait solver to simplify away redundant filters. For example, it correctly deduces `x > 1 AND x > 2 AND x > 3 AND x < 6 AND x < 7 AND x < 8` is the same as `x>3 AND x<6` ```sql create table foo(x int); D insert into foo values (1); D insert into foo values (5); D explain SELECT * from foo where x > 1 AND x > 2 AND x > 3 AND x < 6 AND x < 7 AND x < 8; ┌─────────────────────────────┐ │┌───────────────────────────┐│ ││ Physical Plan ││ │└───────────────────────────┘│ └─────────────────────────────┘ ┌───────────────────────────┐ │ SEQ_SCAN │ │ ──────────────────── │ │ Table: foo │ │ Type: Sequential Scan │ │ Projections: x │ │ Filters: x>3 AND x<6 │ │ │ │ ~1 Rows │ └───────────────────────────┘ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
