Github user wesm commented on the issue:
https://github.com/apache/spark/pull/18655
On DecimalType, I want to point out that we haven't hardened the memory
format and integration tests between Java<->C++ within Arrow. It would be great
if you could help with this -- we ran into a problem in C++ where we needed an
extra sign bit with 16-byte high precision decimals. So we have 3 memory
representations:
- 4 byte decimals (low precision)
- 8 byte
- 16 byte plus sign bitmap
What is Spark's internal memory representation? cc @cpcloud
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
