CurtHagenlocher opened a new issue, #498:
URL: https://github.com/apache/arrow-dotnet/issues/498
### Describe the bug, including details regarding any error messages,
version, and platform.
`ShreddedVariantArrayBuilder.Build` throws `InvalidCastException` whenever a
shred schema has a `Decimal16` typed value and a row shreds into it. Shredding
a variant into a 16-byte decimal column therefore fails for every input; the
size of the value doesn't matter.
Repro, against the Apache.Arrow.Operations 23.0.0 package on .NET 10
(Windows):
```csharp
using System.Data.SqlTypes;
using Apache.Arrow.Operations.Shredding;
using Apache.Arrow.Scalars.Variant;
var schema = ShredSchema.Primitive(ShredType.Decimal16);
// Fits System.Decimal; the failure does not depend on the magnitude.
var values = new List<VariantValue> {
VariantValue.FromDecimal16(SqlDecimal.Parse("1.50")) };
var (metadata, rows) = VariantShredder.Shred(values, schema);
var array = ShreddedVariantArrayBuilder.Build(schema, metadata, rows); //
throws
```
```
System.InvalidCastException: Unable to cast object of type
'System.Data.SqlTypes.SqlDecimal' to type 'System.Decimal'.
at
Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.BuildDecimalArray(ShredType
shredType, IList`1 typedValues, MemoryAllocator allocator)
at
Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.BuildPrimitiveTyped(...)
at
Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.BuildTypedValueArray(...)
at
Apache.Arrow.Operations.Shredding.ShreddedVariantArrayBuilder.Build(...)
```
**Cause** (as of `main` at bb2f7ed). For a `Decimal16` slot,
`VariantShredder` extracts the typed value as a `SqlDecimal`
(`VariantShredder.cs` line 320: `case ShredType.Decimal16: return
value.AsSqlDecimal();`). `BuildDecimalArray` in
`ShreddedVariantArrayBuilder.cs` assumes every value is a `System.Decimal` in
two places:
- it unboxes each value with `b.Append((decimal)v)`, which throws for a
`SqlDecimal`;
- it takes the column's scale from `v is decimal d` values only, so a
`Decimal16` column's scale would come out as 0 even without the cast.
**Expected.** The rows build a `Decimal128` typed column holding the values
at their scale, the same way `Decimal4` and `Decimal8` already work. Values
outside `System.Decimal`'s range (the reason `Decimal16` goes through
`SqlDecimal`) should keep their exact digits. `Decimal128Array.Builder` has an
`Append(SqlDecimal)` overload, which would avoid narrowing through
`System.Decimal`.
As far as I can see there's no test that builds a shredded array with a
`Decimal16` typed value. The existing `Decimal16` tests in
`ShreddedVariantReaderTests` read arrays that were constructed directly.
Found while adding variant statistics to EngineeredWood's Delta writer
(clast-project/engineered-wood#676).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
