adamreeve commented on code in PR #43269:
URL: https://github.com/apache/arrow/pull/43269#discussion_r1678770560
##########
csharp/src/Apache.Arrow/Ipc/ArrowStreamReaderImplementation.cs:
##########
@@ -132,7 +132,13 @@ protected ReadResult ReadMessage()
Flatbuf.Message message =
Flatbuf.Message.GetRootAsMessage(CreateByteBuffer(messageBuff));
- int bodyLength = checked((int)message.BodyLength);
+ if (message.BodyLength > int.MaxValue)
Review Comment:
I don't think there's a need for more specific checks when importing
individual arrays as this should cover all scenarios.
There's also an `ArrowMemoryReaderImplementation` but that needs to be
constructed with a `ReadOnlyMemory<byte>` so is already limited to 2 GiB.
I tested reading data from Flight which uses a
`RecordBatchReaderImplementation`, but that actually fails on the C++ server
side with an error that > 2 GiB record batches are not supported (see
[FlightPayload::Validate](https://github.com/apache/arrow/blob/21238a7f5ebb224410d84ecf32dbe3556cab363a/cpp/src/arrow/flight/types.cc#L207)),
and I'm not sure whether any other languages do support > 2 GiB batches over
Flight.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
