[
https://issues.apache.org/jira/browse/ORC-209?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119185#comment-16119185
]
Gopal V commented on ORC-209:
-----------------------------
I'll submit a PR for the encodings table and document the decimal64
implementation.
> Add Decimal64 Serialization/Deserialization (Part 1)
> ----------------------------------------------------
>
> Key: ORC-209
> URL: https://issues.apache.org/jira/browse/ORC-209
> Project: ORC
> Issue Type: Bug
> Reporter: Matt McCline
> Assignee: Matt McCline
> Priority: Critical
> Fix For: 2.0.0
>
> Attachments: ORC-209.01.wip.patch, ORC-209.02.wip.patch,
> ORC-209.03.patch, storage-api.01.wip.patch, storage-api.02.wip.patch
>
>
> Currently, HiveDecimal is serialized in ORC in a special binary bytes format
> as the "value" stream and a secondary stream with the scale for each decimal.
> The decimal has trailing zeroes removed and the scale can vary for each
> decimal. This format has CPU and storage space (i.e. compression)
> inefficiencies.
> The decimal type has a fixed precision and scale. Gopal/Prasanth/Owen have
> suggested storing the decimals with the trailing zeroes (so the scale is a
> constant value for the file from the metadata) and store it as an integer
> stream that can benefit from run-length encoding compression, etc.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
