asolimando commented on code in PR #21077: URL: https://github.com/apache/datafusion/pull/21077#discussion_r2974092060
########## datafusion/sqllogictest/test_files/parquet_statistics.slt: ########## @@ -59,7 +59,7 @@ query TT EXPLAIN SELECT * FROM test_table WHERE column1 = 1; Review Comment: Good suggestion! I added tests for all numeric types (Int8, Int16, Int32, Int64, UInt32, Float32, Float64), which all work as expected (NDV=Exact(1)). For strings (Utf8), interval analysis does not collapse equality predicates to a single-value interval, because [`next_value_helper()`](https://github.com/apache/datafusion/blob/f734ec54dc1b93b964bd69cfeca725b117f16244/datafusion/expr-common/src/interval_arithmetic.rs#L1233) in `interval_arithmetic.rs` has no implementation for string types (there's no natural "next string" like there is for integers/floats). So `a = 'hello'` does not produce NDV=Exact(1) today. Similarly, temporal types (Timestamp, Interval) don't collapse either despite having `next_value` support - the constraint propagation solver doesn't resolve equality for them. I filed #21109 (strings) and #21111 (temporal types) to track these separately, and renamed this PR to reflect that the optimization is currently limited to numeric types. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
