[
https://issues.apache.org/jira/browse/ARROW-1946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16306623#comment-16306623
]
ASF GitHub Bot commented on ARROW-1946:
---------------------------------------
siddharthteotia commented on issue #1443: ARROW-1946: [JAVA] Add APIs to
decimal vector for writing big endian data
URL: https://github.com/apache/arrow/pull/1443#issuecomment-354519070
@BryanCutler, the in-memory arrow buffer format is LE and Dremio has moved
to LE format as well. However, as part of making the changes in Dremio, we
obviously had to swap bytes in places where we are writing BE data (say from
our Parquet reader code) into buffer of decimal vector through DecimalWriter or
set interfaces.
So in order to optimize this swapping, we created new interface(s) that
allow us to swap bytes while writing itself -- as opposed to allocating a temp
buffer, reading source BE data, writing target LE data into temp buffer and
using the latter to write data into decimal vector.
The new APIs you see in this patch are introduced for this particular
purpose. It's an optimization and not really a necessity. I agree that the API
in general may not be useful but sort of helps our case.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Add APIs to decimal vector for writing big endian data
> ------------------------------------------------------
>
> Key: ARROW-1946
> URL: https://issues.apache.org/jira/browse/ARROW-1946
> Project: Apache Arrow
> Issue Type: Bug
> Components: Java - Vectors
> Reporter: Siddharth Teotia
> Assignee: Siddharth Teotia
> Labels: pull-request-available
>
> We recently moved Dremio to LE Decimal format (similar to Arrow). As part of
> that we introduces some APIs in decimal vector which take a big endian data
> and swap the bytes while writing into the ArrowBuf of decimal vector.
> The advantage of these APIs is that caller would not have to allocate an
> additional memory and write( and read) source big endian twice for swapping
> into new memory and using that to write into the vector.
> We can directly swap bytes while writing into the vector -- just read once
> and swap while writing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
