voonhous opened a new pull request, #20532: URL: https://github.com/apache/hudi/pull/20532
### Describe the issue this Pull Request addresses Closes #20460 Data skipping re-applies `Add`, `Multiply` and `ShiftLeft` with a constant to a column's min/max stats. Without ANSI, integral arithmetic wraps around on overflow, so the max can map below the min (`Long.MaxValue * 2 = -2`) and files holding matching rows are pruned. Decimal results overflow into NULL, which prunes the file too. ### Summary and Changelog All changes are in `ConstantOperandRule`: - `Add` and `Multiply` only match a float or double result. Floating-point arithmetic rounds monotonically and saturates to infinity, so the ordering holds. - The `ShiftLeft` arm is removed: it only takes integral values. - The `Add` constant must also be finite: an infinite addend maps the opposite infinity to NaN, which sorts above every number (`-Infinity + Infinity = NaN`). `Multiply` already rejects infinite factors (#20457). Tests: - `TestDataSkippingUtils`: the three repros from #20460 (bigint `A * 2L`, `A + 1`, `shiftleft(A, 1)`), plus a double `Add` control that still prunes. - `TestHoodieCatalystExpressionUtils`: integral, decimal and `ShiftLeft` cases now return no range; float/double controls still match; infinite and NaN addends are rejected. Cases that used int columns to test other guards (sign, constant operand) now use double columns. - Golden file: the integral and decimal `Add` / `Multiply` / `ShiftLeft` rows now compile to `true`. ### Impact Filters such as `int_col + 1 > 5`, `bigint_col * 2 < 100` or `shiftleft(c, 1) > 0` no longer prune files. Float/double arithmetic filters still prune. ### Risk Level low ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
