zeroshade commented on code in PR #13992:
URL: https://github.com/apache/arrow/pull/13992#discussion_r958810514
##########
go/arrow/array/decimal128.go:
##########
@@ -275,12 +276,12 @@ func (b *Decimal128Builder) unmarshalOne(dec
*json.Decoder) error {
// what got me the closest equivalent values with the values
// that I tested with, and there isn't a good way to push
// an option all the way down here to control it.
- out, _, err = big.ParseFloat(v, 10, 128, big.ToNearestAway)
+ out, _, err = big.ParseFloat(v, 10, 127, big.ToNearestAway)
Review Comment:
The "precision" for the `math/big` library for Golang is actually the
bitwidth of the mantissa, and not the number of digits of precision. In this
case I needed to properly account for the sign-bit because instead of parsing
"0.56" correctly, it was getting "0.55999999999......." and then rounding down
to 0.55, so the tests didn't succeed.
By accounting for the sign-bit correctly, after processing the scale I get
the correct value now
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
