[
https://issues.apache.org/jira/browse/ARROW-7177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liya Fan updated ARROW-7177:
----------------------------
Description:
Vector loading/unloading transforms a set of vectors to and from a set of
buffers with meta data. It is heavily used in flight/IPC.
In the loading/unloading operations, only the number of type buffers are really
needed. However, the current code logic gets a copy of the type buffers, which
is not necessary.
In this issue, we provide a utility to get the number of type buffers, given an
arrow type. It improves the performance because it removes the following
overhead:
1. creating type buffer objects unnecessarily.
2. creating a list and copying list contents (in TypeLayout#getBufferTypes) for
vector unloading.
was:
Vector loading/unloading transforms a set of vectors to and from a set of
buffers with meta data. It is heavily used in flight/IPC.
In the loading/unloading operations, only the number of type buffers are really
needed. However, the current code logic gets a copy of the type buffers, which
is not necessary.
In this issue, we provide a utility to get the number of type buffers, given an
arrow type. It improves the performance by
1. avoiding creating type buffer objects unnecessarily.
2. avoiding creating a list and copying list contents (in
TypeLayout#getBufferTypes) for vector unloading.
> [Java] Provide a utility to improve the performance of vector
> loading/unloading
> -------------------------------------------------------------------------------
>
> Key: ARROW-7177
> URL: https://issues.apache.org/jira/browse/ARROW-7177
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java
> Reporter: Liya Fan
> Assignee: Liya Fan
> Priority: Major
> Labels: pull-request-available
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Vector loading/unloading transforms a set of vectors to and from a set of
> buffers with meta data. It is heavily used in flight/IPC.
> In the loading/unloading operations, only the number of type buffers are
> really needed. However, the current code logic gets a copy of the type
> buffers, which is not necessary.
> In this issue, we provide a utility to get the number of type buffers, given
> an arrow type. It improves the performance because it removes the following
> overhead:
> 1. creating type buffer objects unnecessarily.
> 2. creating a list and copying list contents (in TypeLayout#getBufferTypes)
> for vector unloading.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
