This is an automated email from the ASF dual-hosted git repository.
pitrou pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new fcf8f301422 GH-22342: [Format][Documentation] Clarify Union type ids
(#50839)
fcf8f301422 is described below
commit fcf8f3014227cafe24deeb044c2d51f5fc9902e5
Author: Eric Wang <[email protected]>
AuthorDate: Tue Oct 6 05:25:12 2026 -0400
GH-22342: [Format][Documentation] Clarify Union type ids (#50839)
### Rationale for this change
The Union layout documentation does not explain that the type IDs stored in
the types buffer can differ from the child array indices, or how the optional
`typeIds` metadata maps children to physical type IDs.
### What changes are included in this PR?
- Document the relationship between Union child array indices, physical
type IDs, and the optional `typeIds` metadata.
- Clarify that the types buffer stores the type ID for each slot.
Fixes #22342.
### Are these changes tested?
`git diff --check` passes and the RST change was reviewed against
`Schema.fbs` and the existing Union layout documentation. A local Sphinx/RST
checker is not installed in this environment.
### Are there any user-facing changes?
No.
* GitHub Issue: #22342
Lead-authored-by: erichanwang <[email protected]>
Co-authored-by: Antoine Pitrou <[email protected]>
Signed-off-by: Antoine Pitrou <[email protected]>
---
docs/source/format/Columnar.rst | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/docs/source/format/Columnar.rst b/docs/source/format/Columnar.rst
index adf197887eb..f95d7e80769 100644
--- a/docs/source/format/Columnar.rst
+++ b/docs/source/format/Columnar.rst
@@ -864,6 +864,12 @@ A union is defined by an ordered sequence of types; each
slot in the
union can have a value chosen from these types. The types are named
like a struct's fields, and the names are part of the type metadata.
+Each child type in a union has a logical type id (an 8-bit signed integer)
+that identifies it in the types buffer. By default, these type ids
+correspond directly to the child array indices (0, 1, ...), but an optional
+``typeIds`` metadata array can provide an explicit mapping from child array
+index to type id.
+
Unlike other data types, unions do not have their own validity bitmap. Instead,
the nullness of each slot is determined exclusively by the child arrays which
are composed to create the union.
@@ -878,10 +884,10 @@ Dense union represents a mixed-type array with 5 bytes of
overhead for
each value. Its physical layout is as follows:
* One child array for each type
-* Types buffer: A buffer of 8-bit signed integers. Each type in the
- union has a corresponding type id whose values are found in this
- buffer. A union with more than 128 possible types can be modeled as
- a union of unions.
+* Types buffer: A buffer of 8-bit signed integers, indicating the logical type
+ id of each slot. Note that these type ids are not necessarily the
+ same as the child array index (see above). A union with more than 128
+ possible types can be modeled as a union of unions.
* Offsets buffer: A buffer of signed Int32 values indicating the
relative offset into the respective child array for the type in a
given slot. The respective offsets for each child value array must
