pitrou commented on a change in pull request #11607:
URL: https://github.com/apache/arrow/pull/11607#discussion_r743154574
##########
File path: docs/source/format/Columnar.rst
##########
@@ -693,6 +690,16 @@ Only the slot in the array corresponding to the type index
is considered. All
"unselected" values are ignored and could be any semantically correct array
value.
+.. note:: Critically, the sparse union allows shared columns to be reused
between union members
+ in the ubiquitous union-of-structs with non-overlapping-fields use case.
For example the union::
+
+ SparseUnion<m1: Struct<i: Int32>,
+ m2: Struct<i: Int32, f: Float32, s: VarBinary>,
+ m3: Struct<f: Float32, s: VarBinary>>
+
+ could be backed by just three columns (one for each type) since no union
member requires more
+ than one of each.
Review comment:
Hmm, right. But it wouldn't necessarily be reused on the read side,
unless the reader actively tries to detect this case?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
