waynexia commented on pull request #8856: URL: https://github.com/apache/arrow/pull/8856#issuecomment-739533505
There are some points I'm not very clear about: 1. Ordering. I didn't find a definition of `List` in the documentation or CPP implementation. So I use something like lexicographical order (refer to `sort_list()` in sort.rs) to compare two lists. Please let me know if I missed anything about the ordering's definition. 2. Nulls. It seems that nulls element in list array is different than [documentation](http://arrow.apache.org/docs/format/Columnar.html#variable-size-list-layout). As `From<ArrayData>` requires buffers number should be 1 (0 in `FixedSizeListArray`) which is not the same as the spec. It says that variable sized list has two buffers for "validate bitmap" and "offsets". In the current implementation validate bitmap is placed in `ArrayData`'s `null_bitmap`. I think nullable array in `List` cannot be represented in this way? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
