sdf-jkl opened a new issue, #11309:
URL: https://github.com/apache/arrow-rs/issues/11309
### Describe the bug
`variant_get` can return out-of-precision decimal values when converting
Int32 and Int64 variants. Arrow’s cast kernel rejects the same conversions.
The integer branches call `rescale_decimal`, treating Int32 as a precision-9
decimal and Int64 as precision 18. However, these integers can contain 10 and
19 digits respectively. The rescaler’s fast path relies on the declared source
precision and can skip output precision validation.
This is similar to #10795 for float-to-decimal conversions.
### To Reproduce
```rust
use arrow::array::{ArrayRef, AsArray, Int32Array};
use arrow::compute::cast;
use arrow::datatypes::{DataType, Decimal32Type, DecimalType, Field};
use parquet_variant::Variant;
use parquet_variant_compute::{GetOptions, VariantArrayBuilder, variant_get};
use std::sync::Arc;
let target = DataType::Decimal32(9, 0);
// Arrow cast rejects the value: it exceeds nine digits.
let integers: ArrayRef = Arc::new(Int32Array::from(vec![1_000_000_000]));
assert!(cast(&integers, &target).unwrap().is_null(0));
// variant_get returns an out-of-precision decimal.
let mut builder = VariantArrayBuilder::new(1);
builder.append_variant(Variant::Int32(1_000_000_000));
let input = ArrayRef::from(builder.build());
let options = GetOptions::new()
.with_as_type(Some(Arc::new(Field::new("r", target, true))));
let out = variant_get(&input, options).unwrap();
let out = out.as_primitive::<Decimal32Type>();
assert!(!out.is_null(0));
assert_eq!(out.value(0), 1_000_000_000);
assert!(!Decimal32Type::is_valid_decimal_precision(out.value(0), 9));
```
The same problem occurs with `Variant::Int64(1_000_000_000_000_000_000)`
targeting `Decimal64(18, 0)`. Strict mode also accepts these values instead of
returning an error.
### Expected behavior
Match Arrow’s integer-to-decimal cast: return null on precision overflow in
safe mode and an error in strict mode.
### Additional context
A shared checked integer-to-decimal scalar helper could keep the array cast
and Variant conversion consistent, following the string conversion
consolidation in #10850 and the float conversion work in #11302.
There is also a related negative-scale discrepancy: casting integer `199` to
`Decimal32(3, -1)` produces `190`, while `variant_get` produces `200`. Arrow’s
integer cast truncates, whereas the decimal rescaler used by Variant rounds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
