mapleFU opened a new issue, #8681: URL: https://github.com/apache/arrow-rs/issues/8681
**Describe the bug** This bug comes from this optimization: https://github.com/apache/arrow-rs/pull/7873 The reason is because, view layout, which is in https://arrow.apache.org/docs/format/Columnar.html#variable-size-binary-view-layout , would have only 4bytes for offset. If the sum off offset exceeds i32::MAX, the built array violates the StringView standard, which causing the bug. See https://github.com/apache/arrow-rs/blob/94d51f4efc5e1817062aabad5c6301cbbee2ec61/arrow-array/src/array/byte_view_array.rs#L516 for detail **To Reproduce** Provide input which is longer than 2GiB, or 4GiB, the content would be buggy **Expected behavior** Produce valid data buffer **Additional context** No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
