pitrou commented on a change in pull request #11607:
URL: https://github.com/apache/arrow/pull/11607#discussion_r742933084
##########
File path: docs/source/format/Columnar.rst
##########
@@ -693,6 +690,16 @@ Only the slot in the array corresponding to the type index
is considered. All
"unselected" values are ignored and could be any semantically correct array
value.
+.. note:: Critically, the sparse union allows shared columns to be reused
between union members
+ in the ubiquitous union-of-structs with non-overlapping-fields use case.
For example the union::
+
+ SparseUnion<m1: Struct<i: Int32>,
+ m2: Struct<i: Int32, f: Float32, s: VarBinary>,
+ m3: Struct<f: Float32, s: VarBinary>>
+
+ could be backed by just three columns (one for each type) since no union
member requires more
+ than one of each.
Review comment:
Ah, I understand the statement better now... but is it actually true?
One may hack this in an Arrow implementation, but that wouldn't work for IPC
transfers. This note seems quite confusing actually.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
