nevi-me commented on a change in pull request #7220:
URL: https://github.com/apache/arrow/pull/7220#discussion_r428241219
##########
File path: rust/arrow/src/ipc/writer.rs
##########
@@ -349,8 +350,9 @@ fn write_array_data(
let null_buffer = match array_data.null_buffer() {
None => {
// create a buffer and fill it with valid bits
- let buffer = MutableBuffer::new(num_rows);
- let buffer = buffer.with_bitset(num_rows, true);
+ let num_bytes = bit_util::ceil(num_rows, 8);
Review comment:
@andygrove @pauldix @maxburke this might be of interest for
interop/integration with other implementations. I was overstating the length of
the null buffer by 8. I suspect that with a sufficiently large batch size (like
the 1 million that I tried with), it could cause issues with reading
rust-generated IPC files.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
