SreeramaYeshwanthGowd opened a new pull request, #57476: URL: https://github.com/apache/spark/pull/57476
### What changes were proposed in this pull request? Add a built in `truncate(expr[, scale])` scalar function that truncates a numeric value toward zero to `scale` decimal places. `scale` defaults to 0, and a negative `scale` truncates digits to the left of the decimal point. API surface added: - SQL: `truncate(expr)` and `truncate(expr, scale)` - Scala DataFrame: `functions.truncate(col)`, `functions.truncate(col, scale)` - PySpark, classic and Spark Connect: `pyspark.sql.functions.truncate(col, scale=None)` Implementation notes: - `Truncate` is a small `RoundBase` subclass using `RoundingMode.DOWN`, mirroring the existing `Round` (HALF_UP) and `BRound` (HALF_EVEN). It reuses the same input types, result type derivation, and constant scale handling. - `RoundBase` routes `DecimalType` through `Decimal.changePrecision`, which previously supported only the FLOOR, CEILING, HALF_UP, and HALF_EVEN modes. This adds `Decimal.ROUND_DOWN` and the corresponding branches so the DOWN mode is supported. The branches are trivial: on the compact (long backed) path, integer division already truncates toward zero, so no adjustment is needed; the BigDecimal path already uses `setScale(scale, roundMode)`, which supports DOWN natively. On naming: this adds a new function named `truncate` rather than overloading the existing `trunc`. Spark's `trunc` is date only (`trunc(date, fmt)`), and both the date form and a numeric `trunc(numeric, scale)` take two arguments, so they cannot be distinguished by arity the way `ceil`/`floor` overload their scale argument. Introducing a numeric overload of `trunc` would require type based dispatch at parse time, which is a larger and more error prone change. The name `truncate` matches MySQL and Trino, and the behavior (truncation toward zero) matches PostgreSQL `trunc`, BigQuery `TRUNC`, Oracle, and Snowflake. ### Why are the changes needed? Truncation toward zero to a given number of decimal places is a common numeric operation that Spark cannot express directly today. `floor` and `ceil` with a scale round toward negative and positive infinity, which is wrong for negative values, and `round`/`bround` round to nearest. The function is requested in SPARK-40945 and is provided by PostgreSQL, BigQuery, MySQL, Oracle, Snowflake, and Trino, so it also improves parity with the engines Spark users migrate from. ### Does this PR introduce _any_ user-facing change? Yes. It adds a new built in SQL function `truncate` and the corresponding Scala and PySpark DataFrame API entries. No existing behavior changes; the `Decimal` change only adds support for a new rounding mode and does not alter the existing modes. Example: ``` spark-sql> SELECT truncate(1234.5678, 2); 1234.56 spark-sql> SELECT truncate(-1234.5678, 2); -1234.56 spark-sql> SELECT truncate(1234.5678, -2); 1200 ``` ### How was this patch tested? Added catalyst unit tests in `MathExpressionsSuite` covering decimal (both long backed and BigDecimal backed), double, and integral inputs, positive and negative values, positive and negative scale, the default scale, and null propagation. Extended `DecimalSuite` so its existing "respect rounding mode" test also exercises `ROUND_DOWN` by comparing `Decimal.changePrecision` against `BigDecimal.setScale` for the DOWN mode. Added a DataFrame API test in `MathFunctionsSuite`, PySpark doctests, and regenerated `sql-expression-schema.md`. ### Was this patch authored or co-authored using generative AI tooling? No <!-- Complete this section before submitting the PR. --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
