[
https://issues.apache.org/jira/browse/ARROW-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liya Fan updated ARROW-7177:
----------------------------
Description:
Vector loading/unloading transforms a set of vectors to and from a set of
buffers with meta data. It is heavily used in flight/IPC.
In the loading/unloading operations, only the number of type buffers are really
needed. However, the current code logic gets a copy of the type buffers, which
is not necessary.
In this issue, we provide a utility to get the number of type buffers, given an
arrow type. It improves the performance by
1. avoiding creating type buffer objects unnecessarily.
2. avoiding creating a list and copying list contents (in
TypeLayout#getBufferTypes) for vector unloading.
was:
Vector loading/unloading transforms a set of vectors to and from a set of
buffers with meta data. It is heavily used in flight/IPC.
In the loading/unloading operations, only the number of type buffers are really
needed. However, the current code logic gets a copy of the type buffers, which
is not necessary.
In this issue, we provide a utility to get the number of type buffers, given an
arrow type. It improves the performance by
1. avoiding creating objects unnecessarily.
2. avoiding list copying for vector unloading (which calls
TypeLayout#getBufferTypes).
> [Java] Provide a utility to improve the performance of vector
> loading/unloading
> -------------------------------------------------------------------------------
>
> Key: ARROW-7177
> URL: https://issues.apache.org/jira/browse/ARROW-7177
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
> Reporter: Liya Fan
> Assignee: Liya Fan
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> Vector loading/unloading transforms a set of vectors to and from a set of
> buffers with meta data. It is heavily used in flight/IPC.
> In the loading/unloading operations, only the number of type buffers are
> really needed. However, the current code logic gets a copy of the type
> buffers, which is not necessary.
> In this issue, we provide a utility to get the number of type buffers, given
> an arrow type. It improves the performance by
> 1. avoiding creating type buffer objects unnecessarily.
> 2. avoiding creating a list and copying list contents (in
> TypeLayout#getBufferTypes) for vector unloading.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
