mobiusklein opened a new issue, #250: URL: https://github.com/apache/arrow-dotnet/issues/250
### Describe the bug, including details regarding any error messages, version, and platform. https://github.com/apache/arrow-dotnet/blob/70ab46691df24225e2407ad3cc0d22fd534eb18b/src/Apache.Arrow/C/CArrowArrayImporter.cs#L473 I have a data file which includes very large arrays of 64-bit wide types like `double`. In one case this is `65_651_042` values and the bit width is `64`. Of course, when you multiply them together you get `4_201_666_688` a number too large to fit in a 32-bit signed integer, nearly too big for a 32-bit unsigned integer. After dividing by 8 to convert from bits to bytes again, the value falls within range of a 32-bit integer, but before that can happen, overflow occurs: ``` Unhandled exception: System.AggregateException: One or more errors occurred. (Arithmetic operation resulted in an overflow.) ---> System.OverflowException: Arithmetic operation resulted in an overflow. at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.ImportFixedWidthBuffers(CArrowArray* cArray, Int32 bitWidth) at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.GetAsArrayData(CArrowArray* cArray, IArrowType type) at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.ProcessListChildren(CArrowArray* cArray, IArrowType type) at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.GetAsArrayData(CArrowArray* cArray, IArrowType type) at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.ProcessStructChildren(CArrowArray* cArray, IReadOnlyList`1 fields) at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.GetAsArrayData(CArrowArray* cArray, IArrowType type) at Apache.Arrow.C.CArrowArrayImporter.ImportedArrowArray.GetAsRecordBatch(Schema schema) at Apache.Arrow.C.CArrowArrayImporter.ImportRecordBatch(CArrowArray* ptr, Schema schema) at Apache.Arrow.C.CArrowArrayStreamImporter.ImportedArrowArrayStream.ReadNextRecordBatchAsync(CancellationToken cancellationToken) ``` PR incoming. Platform: Windows 10 (64-bit) dotnet version: 9 Apache.Arrow version: 22.1.0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
