voonhous opened a new pull request, #20759: URL: https://github.com/apache/hudi/pull/20759
### Describe the issue this Pull Request addresses Closes #20567 A bloom filter expression index over a DATE result never matched: the writer adds `toString` of the external value (`2023-09-22`), while the reader probed with the literal's epoch days (`19622`). #20540 stopped the wrong results by not probing such terms, so a DATE filter skipped nothing. ### Summary and Changelog A bloom filter expression index over a DATE result prunes files again. - `ExpressionIndexSupport`: a DATE literal now gives the writer's strings as probe keys. Both the `java.sql.Date` and the `LocalDate` form are probed, since the writer prints one or the other depending on `spark.sql.datetime.java8API.enabled` at write time, and they differ for edge years. A file passes if it may hold any key, so the extra key only keeps files. - TIMESTAMP, floating point and decimal terms still give no keys (reasons in the scaladoc). The writer's key format is unchanged, so existing indexes keep working. - `TestExpressionIndex`: the bloom case that pinned no pruning now expects 1 file instead of 2, plus an equality case; the index is written with the java8 API off and on. ### Impact DATE equality and IN filters can use a bloom filter expression index. ### Risk Level low. Only DATE terms gain probe keys; the test fails without the change. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
