jorisvandenbossche commented on code in PR #41958:
URL: https://github.com/apache/arrow/pull/41958#discussion_r1626228048
##########
docs/source/format/Columnar.rst:
##########
@@ -70,21 +70,131 @@ concepts, here is a small glossary to help disambiguate.
without taking into account any value semantics. For example, a
32-bit signed integer array and 32-bit floating point array have the
same layout.
-* **Parent** and **child arrays**: names to express relationships
- between physical value arrays in a nested type structure. For
- example, a ``List<T>``-type parent array has a T-type array as its
- child (see more on lists below).
+* **Data type**: An application-facing semantic value type that is
+ implemented using some physical layout. For example, Decimal128
+ values are stored as 16 bytes in a fixed-size binary
+ layout. A timestamp may be stored as 64-bit fixed-size layout.
* **Primitive type**: a data type having no child types. This includes
such types as fixed bit-width, variable-size binary, and null types.
* **Nested type**: a data type whose full structure depends on one or
more other child types. Two fully-specified nested types are equal
if and only if their child types are equal. For example, ``List<U>``
is distinct from ``List<V>`` iff U and V are different types.
-* **Logical type**: An application-facing semantic value type that is
- implemented using some physical layout. For example, Decimal
- values are stored as 16 bytes in a fixed-size binary
- layout. Similarly, strings can be stored as ``List<1-byte>``. A
- timestamp may be stored as 64-bit fixed-size layout.
+* **Parent** and **child arrays**: names to express relationships
+ between physical value arrays in a nested type structure. For
+ example, a ``List<T>``-type parent array has a T-type array as its
+ child (see more on lists below).
+* **Parametric type**: a type which requires additional parameters
+ for full determination of its semantics. For example, all nested types
+ are parametric by construction. A timestamp is also parametric as it needs
+ a unit (such as microseconds) and a timezone.
+
+Data Types
+==========
+
+The `Schema.fbs`_ defines built-in data types supported by the
+Arrow columnar format. Each data type uses a well-defined physical layout.
+
+`Schema.fbs`_ is the authoritative source for the description of the
+standard Arrow data types. However, we also provide the below table for
+convenience:
+
++--------------------+------------------------------+------------------------------------------------------------+
+| Type | Type Parameters *(1)* | Physical Memory Layout
|
++====================+==============================+============================================================+
+| Null | | Null
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Boolean | | Fixed-size Primitive
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Int | * bit width | *" (same as above)*
|
+| | * signedness |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Floating Point | * precision | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Decimal | * bit width | *"*
|
+| | * scale |
|
+| | * precision |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Date | * unit | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Time | * bit width *(2)* | *"*
|
+| | * unit |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Timestamp | * unit | *"*
|
+| | * timezone |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Interval | * unit | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Duration | * unit | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Fixed-Size Binary | * byte width | Fixed-size Binary
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Binary | | Variable-size Binary
with 32-bit offsets |
++--------------------+------------------------------+------------------------------------------------------------+
+| Utf8 | | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Large Binary | | Variable-size Binary
with 64-bit offsets |
++--------------------+------------------------------+------------------------------------------------------------+
+| Large Utf8 | | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Binary View | | Variable-size Binary
View |
++--------------------+------------------------------+------------------------------------------------------------+
+| Utf8 View | | *"*
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Fixed-Size List | * *value type* | Fixed-size List
|
+| | * list size |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| List | * *value type* | Variable-size List with
32-bit offsets |
++--------------------+------------------------------+------------------------------------------------------------+
+| Large List | * *value type* | Variable-size List with
64-bit offsets |
++--------------------+------------------------------+------------------------------------------------------------+
+| List View | * *value type* | Variable-size List View
with 32-bit offsets and sizes |
++--------------------+------------------------------+------------------------------------------------------------+
+| Large List View | * *value type* | Variable-size List View
with 64-bit offsets and sizes |
++--------------------+------------------------------+------------------------------------------------------------+
+| Struct | * *children* | Struct
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Map | * *children* | Variable-size List of
Structs |
+| | * keys sortedness |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Union | * *children* | Dense or Sparse Union
*(3)* |
+| | * mode |
|
+| | * type ids |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Dictionary | * *index type* *(4)* | Dictionary Encoded
|
+| | * *value type* |
|
+| | * orderedness |
|
++--------------------+------------------------------+------------------------------------------------------------+
+| Run-End Encoded | * *run end type* *(5)* | Run-End Encoded
|
+| | * *value type* |
|
++--------------------+------------------------------+------------------------------------------------------------+
+
+* \(1) Type parameters listed in *italics* denote a data type's child types.
+
+* \(2) Whether a Union type uses the Sparse or Dense layout is denoted by its
+ *mode* parameter.
+
+* \(3) The *bit width* parameter of a Time type is technically redundant as
+ each *unit* mandates a single bit width.
Review Comment:
2 and 3 should be swapped according to the numbers used in the table
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
