my-vegetable-has-exploded commented on PR #5217: URL: https://github.com/apache/arrow-rs/pull/5217#issuecomment-1862037770
> I think the output should be the union of all the null buffers. I think the nullbuffer for subarrays is only valid for the subarray itself. Take the Example Layout in the documentation as an example(https://arrow.apache.org/docs/format/Columnar.html#struct-layout), if use the union of all the null buffers, the second slot also gets null, which is a little different from my understanding. ``` [{'joe', 1}, {null, 2}, null, {'mark', 4}] * Length: 4, Null count: 1 * Validity bitmap buffer: | Byte 0 (validity bitmap) | Bytes 1-63 | |--------------------------|-----------------------| | 00001011 | 0 (padding) | * Children arrays: * field-0 array (`VarBinary`): * Length: 4, Null count: 2 * Validity bitmap buffer: | Byte 0 (validity bitmap) | Bytes 1-63 | |--------------------------|-----------------------| | 00001001 | 0 (padding) | * Offsets buffer: | Bytes 0-19 | Bytes 20-63 | |----------------|-----------------------| | 0, 3, 3, 3, 7 | unspecified (padding) | * Value buffer: | Bytes 0-6 | Bytes 7-63 | |----------------|-----------------------| | joemark | unspecified (padding) | * field-1 array (int32 array): * Length: 4, Null count: 1 * Validity bitmap buffer: | Byte 0 (validity bitmap) | Bytes 1-63 | |--------------------------|-----------------------| | 00001011 | 0 (padding) | * Value Buffer: | Bytes 0-3 | Bytes 4-7 | Bytes 8-11 | Bytes 12-15 | Bytes 16-63 | |-------------|-------------|-------------|-------------|-----------------------| | 1 | 2 | unspecified | 4 | unspecified (padding) | ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
