cpcloud commented on a change in pull request #11607:
URL: https://github.com/apache/arrow/pull/11607#discussion_r743767094
##########
File path: docs/source/format/Columnar.rst
##########
@@ -693,6 +690,16 @@ Only the slot in the array corresponding to the type index
is considered. All
"unselected" values are ignored and could be any semantically correct array
value.
+.. note:: Critically, the sparse union allows shared columns to be reused
between union members
+ in the ubiquitous union-of-structs with non-overlapping-fields use case.
For example the union::
+
+ SparseUnion<m1: Struct<i: Int32>,
+ m2: Struct<i: Int32, f: Float32, s: VarBinary>,
+ m3: Struct<f: Float32, s: VarBinary>>
+
+ could be backed by just three columns (one for each type) since no union
member requires more
+ than one of each.
Review comment:
I think this note is confusing as well, because it's suggesting that
something that requires a good deal of care can be done without really giving
any details about how one might go about doing it. Personally, I think we
should probably remove the note entirely.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
