hcrosse opened a new issue, #1511:
URL: https://github.com/apache/arrow-go/issues/1511
### Describe the bug, including details regarding any error messages,
version, and platform.
`decimal128.Num.Rescale` panics instead of returning an error when a scale
change exceeds its multiplier table or the result overflows. Zero can panic
too. This reaches `scalar.Decimal128.CastTo` and safe compute casts to decimals
and integers.
`Decimal32.Rescale` and `Decimal64.Rescale` can return wrong values without
an error, reject exact results or panic. `Decimal64.ReduceScaleBy` also returns
wrong values for reductions of 10 or more.
Safe casts between `decimal128` and `decimal256` types can return `invalid:
%!s(<nil>)` instead of the rescaling error. With several failing values, the
message can repeat `invalid:`.
I reproduced this on main at `eb1781e7`, on macOS arm64 with Go 1.26.10.
**Reproduction**
Save this as `main.go` in a new subdirectory of an arrow-go checkout at
`eb1781e7faa2b1f6f7499ded6fce7b86f804f303`, then run `go run .` from that
directory:
```go
package main
import (
"fmt"
"github.com/apache/arrow-go/v18/arrow/decimal"
"github.com/apache/arrow-go/v18/arrow/decimal128"
)
func main() {
fmt.Println(decimal.Decimal64(1).Rescale(0, 10))
fmt.Println(decimal.Decimal32(1410065408).Rescale(10, 0))
fmt.Println(decimal.Decimal64(10000000000).ReduceScaleBy(10, false))
_, err := decimal128.FromI64(12).Rescale(0, 38)
fmt.Println(err)
}
```
Actual output before the stack trace:
```text
1410065408 <nil>
1 <nil>
7
panic: arrow/decimal128: cannot represent value larger than 128bits
```
Expected output:
```text
10000000000 <nil>
0 rescale data loss
1
rescale data loss
```
**Cause**
`decimal128.Num.Rescale` indexes its multiplier table without checking the
scale difference and converts an overflowing product with `FromBigInt`. The
scale difference can also overflow `int32`.
https://github.com/apache/arrow-go/blob/eb1781e7faa2b1f6f7499ded6fce7b86f804f303/arrow/decimal128/decimal128.go#L483-L524
The narrow types compute powers of ten in `int32`, including for
`Decimal64`. Their scale differences can wrap too.
https://github.com/apache/arrow-go/blob/eb1781e7faa2b1f6f7499ded6fce7b86f804f303/arrow/decimal/decimal.go#L471-L496
https://github.com/apache/arrow-go/blob/eb1781e7faa2b1f6f7499ded6fce7b86f804f303/arrow/decimal/decimal.go#L405
The decimal cast kernels format their error output instead of the error
returned by `Rescale`.
https://github.com/apache/arrow-go/blob/eb1781e7faa2b1f6f7499ded6fce7b86f804f303/arrow/compute/internal/kernels/numeric_cast.go#L310-L375
I have a patch with tests and can open a PR.
### Component(s)
Other
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
