Just remembered that case (1) started a ML thread some months ago, in case someone else recalls, here is the link:
https://lists.apache.org/thread/7gbqjwykh1ob3xbvwph3ljsdl5c7kxpd El mar, 21 jul 2026 a las 17:48, Antoine Pitrou (<[email protected]>) escribió: > > > Hello, > > https://github.com/apache/arrow/issues/50515 and its associated PR > https://github.com/apache/arrow/pull/50546 raise the question of what > the `nullable` flag in the IPC format means exactly for non-trivial fields. > > For example: > > 1) a non-nullable child field in a struct with some nulls in the parent > (implying that some child elements are logically null) > 2) a dictionary array with nulls in the dictionary (but no nulls in the > indices) > 3) a union or run-end-encoded array with nulls in the child values > > Arrow C++ has historically not been very considerate of the `nullable` > flag internally, treating it as mere metadata with no semantic implications. > > What do other implementations do / what do you suggest implementations > *should* do? > > Regards > > Antoine. >
