0z5a opened a new pull request, #1574: URL: https://github.com/apache/arrow-go/pull/1574
### Rationale for this change `decimal256.FromFloat32` scales the input in float64 and then converts the scaled value back to float32 before splitting it into 64-bit words. A decimal256 can hold values far above `math.MaxFloat32` (about 3.4e38). When the scaled value is larger than that, the float32 conversion gives +Inf, the words are built from `uint64(+Inf)`, and the function returns a garbage value with a nil error. With the `assert` build tag it panics in a `debug.Assert` instead. This can be reached from a safe cast. Results for float32 inputs, on main, with this change, and from pyarrow: | cast | main | this PR | pyarrow | | --- | --- | --- | --- | | `123.5` to `decimal256(76, 38)` | `-62771017353866807638.35789423207666416102355444464034512896` | `123.50000000000000260327792243959757012992` | `123.50000000000000000000000000000000000000` | | `3.4e38` to `decimal256(50, 5)` | `-62771017353866807638357894232076664161023554444640345.12896` | `339999995214436424907732413799364296704.00000` | `339999995214436424907732413799364296704.00000` | | `1e30` to `decimal256(60, 20)` | `-62771017353866807638357894232076664161.02355444464034512896` | `1000000015047466202704776911698.34746997528420417536` | `1000000015047466219876688855040.00000000000000000000` | With this change these values are accurate to float64 precision. pyarrow uses an exact conversion, so its digits can differ past about the 16th significant digit. Only decimal256 is affected. Decimal128, decimal32 and decimal64 check the scaled value against their precision before narrowing it, and their largest values are below the float32 maximum. ### What changes are included in this PR? When the scaled value does not fit in a float32, `fromPositiveFloat32` now splits it as a float64. The word split is moved out of `fromPositiveFloat64` into a small `splitPositiveFloat64` helper that both functions use. Values whose scaled form fits in a float32 still go through the existing float32 path, so their results, including the existing float32 rounding, do not change. I compared the old and new code on about 300,000 random float32 values and scales where the scaled value fits in a float32, and every result was identical. ### Are these changes tested? Yes. A new `TestDecimalFromFloat/float32/scaled_past_float32_range` subtest round-trips float32 values whose scaled form is past the float32 range. It covers positive and negative values up to `math.MaxFloat32`, plus one control value that stays in range. The subtest fails on main (wrong values by default, and a panic with `-tags assert,test`) and passes with this change. `go test -short ./arrow/decimal256/... ./arrow/decimal128/... ./arrow/decimal/... ./arrow/compute/... ./arrow/scalar/...` passes with and without `-tags assert,test`. ### Are there any user-facing changes? Yes. `decimal256.FromFloat32`, and casts from float32 to decimal256, now return a value accurate to float64 precision instead of a garbage value when the scaled input is larger than the float32 range. This change was drafted with Claude Code (claude-opus-5-5); the test results above are from the runs described. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
