ozenyunus35 opened a new pull request, #26346:
URL: https://github.com/apache/datafusion/pull/26346
## Which issue does this PR close?
- Closes #26337.
## Rationale for this change
Spark's `format_string` uses Java's `Formatter`, and `datafusion-spark`
differed from it in two ways: `%s` of a DECIMAL printed the unscaled integer
(`123456` for `123.456`), and `%g` / `%G` dropped trailing zeros like C
(`1234.5`), while Java always prints the requested number of significant digits
(`1234.50`).
## What changes are included in this PR?
- `%s` / `%S` of a `Decimal128` or `Decimal256` prints the value with its
scale in plain notation (`123.456`, `0.50`), through a small
`decimal_to_plain_string` helper.
- `%g` / `%G` no longer strip trailing zeros, for float and decimal
arguments. The `strip_trailing_0s` flag and `trim_trailing_0s` were only used
for this, so they are removed.
- While adding cases, `%g` of `0.0` panicked with "attempt to subtract with
overflow", because the exponent came from `log10(0)`. A `decimal_exponent`
helper returns 0 for zero, so `%g` gives `0.00000` and `%e` gives
`0.000000e+00`, as in Java.
I checked every new expected value by running `String.format` with the same
pattern on JDK 21 (`BigDecimal` for the decimal `%s` cases).
One thing I left as is: Java's `BigDecimal.toString` switches to E notation
for very small values (`1E-7` for `0.0000001`); this PR prints those in plain
notation. I couldn't confirm what Spark's own `Decimal.toString` gives there
without a Spark install, so I didn't guess.
## What is the testing strategy for this PR?
- `format_string.slt`: the expectations the issue points at are updated
(`%s` / `%S` of `Decimal128` and `Decimal256`, `%g` / `%G` of `Float64`,
`Float32`, `Decimal128` and `Decimal256`), and new cases cover `%g` on small,
large, zero, negative and rounded values, `%.3g`, `%.0g`, `%10.2g`, `%e` of
zero, and `%s` / `%S` / `%10s` of decimals. On `main` the file fails (and the
`%g` of `0.0` case panics).
- `cargo test -p datafusion-spark --lib format_string` and `cargo clippy -p
datafusion-spark --all-targets -- -D warnings` pass.
## Are there any user-facing changes?
Yes: `format_string('%s', <decimal>)` prints the decimal with its scale,
`%g` / `%G` keep trailing zeros, and `%g` of zero no longer panics.
I used an AI assistant to help draft parts of the code and tests; I reviewed
the changes and ran the tests myself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
