This is an automated email from the ASF dual-hosted git repository.
Fokko pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-format.git
The following commit(s) were added to refs/heads/master by this push:
new bb22d01 Allow TimestampType to annotate FLBA(12) (#601)
bb22d01 is described below
commit bb22d0171b47000308e24209db79876b8dbe9566
Author: Divjot Arora <[email protected]>
AuthorDate: Fri Sep 11 15:01:44 2026 +0200
Allow TimestampType to annotate FLBA(12) (#601)
---
LogicalTypes.md | 43 ++++++++++++++++++++++++++++--------------
src/main/thrift/parquet.thrift | 2 +-
2 files changed, 30 insertions(+), 15 deletions(-)
diff --git a/LogicalTypes.md b/LogicalTypes.md
index 8ec8f5b..559030f 100644
--- a/LogicalTypes.md
+++ b/LogicalTypes.md
@@ -360,11 +360,18 @@ as shown below.
### TIMESTAMP
-In data annotated with the `TIMESTAMP` logical type, each value is a single
-`int64` number that can be decoded into year, month, day, hour, minute, second
-and subsecond fields using calculations detailed below. Please note that a
value
-defined this way does not necessarily correspond to a single instant on the
-time-line and such interpretations are allowed on purpose.
+In data annotated with the `TIMESTAMP` logical type, each value is an `int64`
+or a 12-byte `FIXED_LEN_BYTE_ARRAY` that can be decoded into year, month, day,
+hour, minute, second and subsecond fields using calculations detailed below.
+Please note that a value defined this way does not necessarily correspond to a
+single instant on the time-line and such interpretations are allowed on
purpose.
+
+For the `int64` carrier, the value is a signed 64-bit integer count of `unit`s
+since the Unix epoch.
+
+For the `FIXED_LEN_BYTE_ARRAY` carrier (with `type_length = 12`), the value is
a
+signed 96-bit two's-complement little-endian integer count of `unit`s since the
+Unix epoch.
The `TIMESTAMP` type has two type parameters:
- `isAdjustedToUTC` must be either `true` or `false`.
@@ -451,12 +458,14 @@ local timestamp in reality.
#### Common considerations
-Every possible `int64` number represents a valid timestamp, but depending on
the
-precision, the corresponding year may be outside of the practical everyday
-limits and implementations may choose to only support a limited range.
+Every possible `int64` and `FIXED_LEN_BYTE_ARRAY(12)` value represents a valid
+timestamp, but depending on the precision, the corresponding year may be
outside
+of the practical everyday limits and implementations may choose to only support
+a limited range.
On the other hand, not every combination of year, month, day, hour, minute,
-second and subsecond values can be encoded into an `int64`. Most notably:
+second and subsecond values can be encoded into an `int64` or
+`FIXED_LEN_BYTE_ARRAY(12)`. Most notably:
- An arbitrary combination of timestamp fields cannot be encoded as a single
number if the values for some of the fields are outside of their normal range
@@ -467,13 +476,19 @@ second and subsecond values can be encoded into an
`int64`. Most notably:
- minute = 61
- month = 13
- day = 29, month = 2, year = any non-leap year
-- Due to the range of the `int64` type, timestamps using the `NANOS` unit
+- Due to the range of the `int64` type, `int64` timestamps using the `NANOS`
unit
can only represent values between 1677-09-21 00:12:43 and 2262-04-11
23:47:16.
- Values outside of this range cannot be represented with the `NANOS`
- unit. (Other precisions have similar limits but those are outside of the
- domain for practical everyday usage.)
+ Values outside of this range should instead be represented with the
+ `FIXED_LEN_BYTE_ARRAY(12)` physical type using the `NANOS` unit. Other
precisions
+ have similar limits but those are outside of the domain for practical
everyday use.
+
+The sort order used for `TIMESTAMP` is signed:
-The sort order used for `TIMESTAMP` is signed.
+- For the `int64` carrier: signed integer comparison.
+- For the `FIXED_LEN_BYTE_ARRAY(12)` carrier: signed two's-complement
comparison of
+ the represented value. The correct ordering can be produced by flipping the
+ most-significant bit of the last byte and then using unsigned byte-wise
comparison
+ starting from the last byte.
#### Deprecated timestamp ConvertedType
diff --git a/src/main/thrift/parquet.thrift b/src/main/thrift/parquet.thrift
index bb58201..fea13fb 100644
--- a/src/main/thrift/parquet.thrift
+++ b/src/main/thrift/parquet.thrift
@@ -365,7 +365,7 @@ union TimeUnit {
/**
* Timestamp logical type annotation
*
- * Allowed for physical types: INT64
+ * Allowed for physical types: INT64 and FIXED_LEN_BYTE_ARRAY with
`type_length` = 12.
*/
struct TimestampType {
1: required bool isAdjustedToUTC
