[
https://issues.apache.org/jira/browse/ARROW-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834451#comment-16834451
]
Jacques Nadeau commented on ARROW-5207:
---------------------------------------
To clarify, are you wanting to reuse the same ArrowBuf within a Vector? Vectors
are already reusable (with different underlying memory) If so, maybe we should
introduce this in a different way where it is purpose built. E.g. reuseMemory.
For example, during reuse, there is some types of memory you need to clear.
Also, I don't understand why you have setting valuecount and datalength. If
you're reusing, you need to fit within whatever you have and then use
setValueCount at the end. The reason we should make this more formal is because
Vectors are mostly thought about as created, sealed, read. This isn't formally
enforced by the API but I think we should try to keep the spirit of this in new
APIs we introduce. If your pattern is different, we should formallize it
specifically as reuse rather than something more low-level as your current PR
proposes.
> [Java] add APIs to support vector reuse
> ---------------------------------------
>
> Key: ARROW-5207
> URL: https://issues.apache.org/jira/browse/ARROW-5207
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java
> Reporter: Ji Liu
> Assignee: Ji Liu
> Priority: Minor
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In some scenarios we hope that ValueVector could be reused to reduce creation
> overhead. This is very common in shuffle stage, it's no need to create
> ValueVector or realloc buffers every time, suppose that the recordCount of
> ValueVector and capacity of its buffers is written in stream, when we
> deserialize it, we can simply judge whether realloc is needed through
> dataLength.
> My proposal is that add APIs in ValueVector to process this logic, otherwise
> users have to implement by themselves if they want to reuse which is not
> user-friendly.
> If you agree with this, I would like to take this ticket. Thanks
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
