0z5a opened a new pull request, #1574:
URL: https://github.com/apache/arrow-go/pull/1574

   ### Rationale for this change
   
   `decimal256.FromFloat32` scales the input in float64 and then converts the 
scaled value back to float32 before splitting it into 64-bit words. A 
decimal256 can hold values far above `math.MaxFloat32` (about 3.4e38). When the 
scaled value is larger than that, the float32 conversion gives +Inf, the words 
are built from `uint64(+Inf)`, and the function returns a garbage value with a 
nil error. With the `assert` build tag it panics in a `debug.Assert` instead.
   
   This can be reached from a safe cast. Results for float32 inputs, on main, 
with this change, and from pyarrow:
   
   | cast | main | this PR | pyarrow |
   | --- | --- | --- | --- |
   | `123.5` to `decimal256(76, 38)` | 
`-62771017353866807638.35789423207666416102355444464034512896` | 
`123.50000000000000260327792243959757012992` | 
`123.50000000000000000000000000000000000000` |
   | `3.4e38` to `decimal256(50, 5)` | 
`-62771017353866807638357894232076664161023554444640345.12896` | 
`339999995214436424907732413799364296704.00000` | 
`339999995214436424907732413799364296704.00000` |
   | `1e30` to `decimal256(60, 20)` | 
`-62771017353866807638357894232076664161.02355444464034512896` | 
`1000000015047466202704776911698.34746997528420417536` | 
`1000000015047466219876688855040.00000000000000000000` |
   
   With this change these values are accurate to float64 precision. pyarrow 
uses an exact conversion, so its digits can differ past about the 16th 
significant digit.
   
   Only decimal256 is affected. Decimal128, decimal32 and decimal64 check the 
scaled value against their precision before narrowing it, and their largest 
values are below the float32 maximum.
   
   ### What changes are included in this PR?
   
   When the scaled value does not fit in a float32, `fromPositiveFloat32` now 
splits it as a float64. The word split is moved out of `fromPositiveFloat64` 
into a small `splitPositiveFloat64` helper that both functions use.
   
   Values whose scaled form fits in a float32 still go through the existing 
float32 path, so their results, including the existing float32 rounding, do not 
change. I compared the old and new code on about 300,000 random float32 values 
and scales where the scaled value fits in a float32, and every result was 
identical.
   
   ### Are these changes tested?
   
   Yes. A new `TestDecimalFromFloat/float32/scaled_past_float32_range` subtest 
round-trips float32 values whose scaled form is past the float32 range. It 
covers positive and negative values up to `math.MaxFloat32`, plus one control 
value that stays in range. The subtest fails on main (wrong values by default, 
and a panic with `-tags assert,test`) and passes with this change. `go test 
-short ./arrow/decimal256/... ./arrow/decimal128/... ./arrow/decimal/... 
./arrow/compute/... ./arrow/scalar/...` passes with and without `-tags 
assert,test`.
   
   ### Are there any user-facing changes?
   
   Yes. `decimal256.FromFloat32`, and casts from float32 to decimal256, now 
return a value accurate to float64 precision instead of a garbage value when 
the scaled input is larger than the float32 range.
   
   This change was drafted with Claude Code (claude-opus-5-5); the test results 
above are from the runs described.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to