[
https://issues.apache.org/jira/browse/PARQUET-758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17597640#comment-17597640
]
ASF GitHub Bot commented on PARQUET-758:
----------------------------------------
gszadovszky commented on PR #184:
URL: https://github.com/apache/parquet-format/pull/184#issuecomment-1231284535
> It isn't clear to me if this should be a logical type or a physical type.
We would need understand if there is different handling for forward
compatibility purposes (what do we want the desired behavior to be be). I think
C++ might be lenient here, but don't know about parquet-mr @gszadovszky
thoughts?
I think the basic idea behind having physical and logical types is to
support forward compatibility since we can always represent (somehow) a
long-existing physical type while logical types are getting extended.
Parquet-mr should work fine with "unknown" logical types by reading it back as
an un-annotated physical vale (a `Binary` with two bytes in this case).
So, if the community supports having a half-precision floating point type I
would vote on specifying it as a logical type.
The tricky thing will be the implementations. Even though parquet-mr does
not really care about converting the values according to their logical types we
still need to care about the logical types at the ordering (min/max values in
the statistics). It would not be too easy to implement the half-precision
floating point comparison logic since java does not have such a primitive type.
(BTW the sorting order of floating point numbers are still an open issue:
[PARQUET-1222](https://issues.apache.org/jira/browse/PARQUET-1222))
> [Format] HALF precision FLOAT Logical type
> ------------------------------------------
>
> Key: PARQUET-758
> URL: https://issues.apache.org/jira/browse/PARQUET-758
> Project: Parquet
> Issue Type: Improvement
> Components: parquet-format
> Reporter: Julien Le Dem
> Priority: Minor
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
