bkietz commented on code in PR #35628:
URL: https://github.com/apache/arrow/pull/35628#discussion_r1204543845
##########
format/Message.fbs:
##########
@@ -99,6 +99,12 @@ table RecordBatch {
/// Optional compression of the message body
compression: BodyCompression;
+
+ /// Some types such as Utf8View are represented using a variable number of
buffers.
+ /// For each such Field in the pre-ordered flattened logical schema, there
will be
+ /// an entry in variadicCounts to indicate the number of extra buffers which
belong
+ /// to that Field.
+ variadicCounts: [long];
Review Comment:
Unless I'm mistaken, this change is backwards compatible with V5. If a
pre-Utf8View V5 reader encounters a schema including a field of Utf8View, I'd
expect it to recognize that as a type it's unprepared to deal with and fail
gracefully from there (before any record batches needed to be read correctly).
For comparison, we didn't bump this for REE
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
