jorisvandenbossche commented on pull request #12557: URL: https://github.com/apache/arrow/pull/12557#issuecomment-1061528900
> A list array has three pieces, values, offsets, and validity. It isn't clear why the offsets would be expected to contain the validity. I would think it just as likely someone assumes the values contains the validity. While "a list array has three pieces, values, offsets, and validity" is of course correct, I think many people will think about (or explain) a list array as consisting of two pieces: values and offsets (those are also the two "child" arrays for which we have properties on ListArray to access them, and are the two arrays from which you can recreate a new ListArray in from_arrays). So I don't think the confusion from ARROW-15837 is that uncommon, and the clarification here seems helpful IMO. Since the values array doesn't have a 1:1 relationship with the *list* values (and can have nulls itself, independent from nulls at the list level), I would find it less expected to think that those would contain the list validity. [ARROW-15839](https://issues.apache.org/jira/browse/ARROW-15839) will indeed help, but then it's maybe also the question if we want to make it easier to get the "mask" / validity bitmap of an existing ListArray (although that's not specific to a ListArray). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
