[
https://issues.apache.org/jira/browse/ARROW-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16101976#comment-16101976
]
Phillip Cloud commented on ARROW-786:
-------------------------------------
That should be possible.
Though because boost is using 128 bits + a sign bit, going from arrow-cpp to
Java won't be possible in every case since the boost representation's values
range from {{+/-0..2 ** 128 - 1}}, whereas the Java implementation's values
range from {{-2 ** 127..2 ** 127 - 1}}.
The more I think about this and reread the boost multiprecision docs, I think
we should just implement our own very small wrapper around native types.
Boost multiprecision has some optimizations that arrow doesn't care about like
this that increase implementation complexity at best and hurt performance at
worst:
{code}
When used at fixed precision, the size of this type is always one machine word
larger than you would expect for an N-bit integer: the extra word stores both
the sign, and how many machine words in the integer are actually in use.
{code}
plus the complexities of have two signed integer representations are enough to
make me want to try jettisoning boost multiprecision.
> [Format] In-memory format for 128-bit Decimals, handling of sign bit
> --------------------------------------------------------------------
>
> Key: ARROW-786
> URL: https://issues.apache.org/jira/browse/ARROW-786
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Format
> Reporter: Wes McKinney
> Fix For: 0.6.0
>
>
> cc [~cpcloud]
> We found in ARROW-655 that we needed to add an extra bit for signedness for
> decimals stored as 128-bit values to be able to use the Boost multiprecision
> libraries. This makes Decimal128 not fit completely neatly as a 16-byte fixed
> size binary value, and more of a {{struct<sign_bitmap: boolean, data:
> fixed_size_binary(16)>}}. What is the current formata in the Java
> implementation? We will need to document the memory layout for decimals that
> maximizes compatibility across languages and eventually implement integration
> tests for IPC.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
