vb-dbrks opened a new pull request, #59408: URL: https://github.com/apache/spark/pull/59408
### What changes were proposed in this pull request? Allow scaled `floor` and `ceil` to retain Float/Double input until directional rounding, avoiding the implicit decimal cast that can move an input across a rounding boundary. A shared internal `RoundToDecimal` implementation preserves the existing decimal result precision/scale and input overflow limits. Decimal inputs, explicit casts and integral-input coercion retain their existing behavior. Generated and interpreted floating-point evaluation use the same rounding helper. No public API, configuration or dependency changes are proposed. JIRA: [SPARK-60202](https://issues.apache.org/jira/browse/SPARK-60202). ### Why are the changes needed? I found and reported this issue while investigating rounding behavior. For Double input, the current implicit `decimal(30,15)` cast rounds before `floor`/`ceil`. For example: ```sql SELECT floor(CAST('0.9999999999999999' AS DOUBLE)), floor(CAST('0.9999999999999999' AS DOUBLE), 0), ceil(CAST('0.9000000000000001' AS DOUBLE), 1); ``` The results change from `(0, 1, 0.9)` to `(0, 0, 1.0)`. The old scaled results violate the rounding direction for those supplied inputs. Runtime columns exhibit the same issue. ### Does this PR introduce _any_ user-facing change? Yes. Scaled floor/ceil now preserve these Float/Double rounding boundaries. For example, `floor(CAST('0.8999999999999999' AS DOUBLE), 1)` changes from `0.9` to `0.8`. Float behavior is corrected too: `floor(CAST('3.1415' AS FLOAT), 4)` changes from `3.1415` to `3.1414`, reflecting the stored Float value `3.141499996185302734375`. Existing decimal result types, maximum output scales, explicit decimal casts, null/non-finite handling and ANSI overflow behavior are preserved. This does not change unary floor/ceil or introduce an exact-binary-decimal arithmetic contract. The reported Double issue was reproduced in released Spark 4.0.2 and 4.1.2 using official artifacts, and in current-master regression tests. ### How was this patch tested? Baseline: Apache master `f04e6d60ebf9131fd83f306627697370ef52a309`, Temurin JDK 21.0.5+11-LTS, macOS ARM64. Two new Double boundary tests failed against unchanged production source before applying the fix. With the fix, these four suites passed, totaling 242 tests: ```sh ./build/sbt \ 'catalyst/testOnly org.apache.spark.sql.catalyst.expressions.MathExpressionsSuite org.apache.spark.sql.catalyst.analysis.TypeCoercionSuite org.apache.spark.sql.catalyst.analysis.AnsiTypeCoercionSuite' \ 'sql/testOnly org.apache.spark.sql.MathFunctionsSuite' ``` New coverage includes positive/negative Float and Double boundaries, small values, negative/zero/positive scales, result types, explicit decimal casts, nulls, signed zero, NaN, infinities and overflow. SQL integration tests exercise literals and runtime columns with ANSI and code generation on/off. Three existing Float floor expectations were corrected after independently checking the represented value. `catalyst/scalastyle`, `catalyst/Test/scalastyle` and `sql/Test/scalastyle` all passed after a whitespace-only correction to one overlong line. Local tests used a temporary Netty 4.2.18 override because the configured proxy blocks 4.2.19; no dependency files changed. Validation with master's default dependencies is pending CI. The SQL suite passed but emitted a possible daemon-thread-leak warning. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (tool version unavailable). I discovered and reported the bug. I used Codex as a coding agent to help implement the fix, add regression tests, run local validation and prepare this description. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
