adamreeve commented on issue #40517:
URL: https://github.com/apache/arrow/issues/40517#issuecomment-1996171032
I initially thought that the problem was just that the files were bigger
than they need to be, and that the array offsets would be correctly
round-tripped. But on further investigation, it looks like the IPC files
written from sliced arrays are invalid and can't be read back by the C# library
or Python library.
Reading these sliced batches back from C# fails with:
```
System.IO.InvalidDataException
Null count length must be >= 0
at Apache.Arrow.Ipc.ArrowReaderImplementation.LoadField(MetadataVersion
version, RecordBatchEnumerator& recordBatchEnumerator, Field field, FieldNode&
fieldNode, ByteBuffer bodyData, IBufferCreator bufferCreator)
```
and Python displays the arrays as invalid:
```
<Invalid array: Buffer #0 too small in array of type int32 and length 100:
expected at least 13 byte(s), got 0>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
