This is an automated email from the ASF dual-hosted git repository.
gabor pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/parquet-format.git
The following commit(s) were added to refs/heads/master by this push:
new 46cc3a0 PARQUET-758: Add Float16/Half-float logical type (#184)
46cc3a0 is described below
commit 46cc3a0647d301bb9579ca8dd2cc356caf2a72d2
Author: Anja Kefala <[email protected]>
AuthorDate: Fri Oct 27 00:01:29 2023 -0700
PARQUET-758: Add Float16/Half-float logical type (#184)
Type involves a trade-off of reduced precision,
in exchange for more efficient storage.
---
LogicalTypes.md | 10 ++++++++++
src/main/thrift/parquet.thrift | 2 ++
2 files changed, 12 insertions(+)
diff --git a/LogicalTypes.md b/LogicalTypes.md
index b387b89..395e8fc 100644
--- a/LogicalTypes.md
+++ b/LogicalTypes.md
@@ -245,6 +245,16 @@ comparison.
To support compatibility with older readers, implementations of parquet-format
should
write `DecimalType` precision and scale into the corresponding SchemaElement
field in metadata.
+### FLOAT16
+
+The `FLOAT16` annotation represents half-precision floating-point numbers in
the 2-byte IEEE little-endian format.
+
+Used in contexts where precision is traded off for smaller footprint and
potentially better performance.
+
+The primitive type is a 2-byte fixed length binary.
+
+The sort order for `FLOAT16` is signed (with special handling of NANs and
signed zeros); it uses the same
[logic](https://github.com/apache/parquet-format#sort-order) as `FLOAT` and
`DOUBLE`.
+
## Temporal Types
### DATE
diff --git a/src/main/thrift/parquet.thrift b/src/main/thrift/parquet.thrift
index 9f90572..e52675e 100644
--- a/src/main/thrift/parquet.thrift
+++ b/src/main/thrift/parquet.thrift
@@ -242,6 +242,7 @@ struct MapType {} // see LogicalTypes.md
struct ListType {} // see LogicalTypes.md
struct EnumType {} // allowed for BINARY, must be encoded with UTF-8
struct DateType {} // allowed for INT32
+struct Float16Type {} // allowed for FIXED[2], must encoded raw FLOAT16 bytes
/**
* Logical type to annotate a column that is always null.
@@ -355,6 +356,7 @@ union LogicalType {
12: JsonType JSON // use ConvertedType JSON
13: BsonType BSON // use ConvertedType BSON
14: UUIDType UUID // no compatible ConvertedType
+ 15: Float16Type FLOAT16 // no compatible ConvertedType
}
/**
