minni31 opened a new pull request, #12775: URL: https://github.com/apache/gluten/pull/12775
## What changes are proposed in this pull request? This PR offloads Spark's `format_number(numeric, int)` function to the Velox native implementation via Gluten. Native offload is applied for integer and floating-point inputs (Byte/Short/Int/Long/Float/Double), which match Velox's registered `format_number` signatures. Two overloads are intentionally **not** supported natively and always fall back to vanilla Spark: - **Format-pattern STRING second argument** (e.g. `format_number(x, '#,##0.00')`) — Velox only implements the integer decimal-places second argument, not the Java `DecimalFormat` pattern overload. - **DecimalType input** — requires full-precision `BigDecimal` formatting; casting to `Double` would lose precision for values with more than 15 significant digits. Fallback is achieved by throwing `GlutenNotSupportException` from `genFormatNumberTransformer`; the validation path (`ValidatablePlan.failValidationWithException`) converts this into a graceful fallback to vanilla Spark, so no query can crash on an unsupported overload. The feature is controlled by a new config `spark.gluten.sql.columnar.formatNumber` (default `true`). Wiring summary: - `ExpressionNames` / `ExpressionMappings`: register `Sig[FormatNumber]`. - `ExpressionConverter`: dispatch `FormatNumber` to `genFormatNumberTransformer`. - `SparkPlanExecApi`: default hook throws `GlutenNotSupportException` so backends without native support fall back cleanly. - `VeloxSparkPlanExecApi`: override with the config gate plus `StringType` / `DecimalType` fallbacks. ## How was this patch tested? Added integration tests in `VeloxStringFunctionsSuite` (compared against vanilla Spark via `runQueryAndCompare`): - Native offload verified for int, bigint, float, and double inputs, and for zero decimal places. Each test asserts a `ProjectExecTransformer` actually carries the `FormatNumber` expression (not merely that some transformer exists in the plan). - Fallback verified for the string format-pattern overload, DecimalType input, and when the config kill-switch is disabled. Each fallback test asserts `FormatNumber` is absent from any `ProjectExecTransformer` and present in a vanilla `ProjectExec`. ## Was this patch authored or co-authored using generative AI tooling? Generated-by: GitHub Copilot -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
