[
https://issues.apache.org/jira/browse/ARROW-1627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16187254#comment-16187254
]
ASF GitHub Bot commented on ARROW-1627:
---------------------------------------
GitHub user yufeldman opened a pull request:
https://github.com/apache/arrow/pull/1150
ARROW-1627: New class to handle collection of BufferLedger(s) within …
…AllocationManager
LowCostIdentityHashMap uses array only to store objects since key
(BaseAllocator) is already included in the value (BufferLedger)
To make this class a bit more generic it uses values that implement
ValueWithKeyIncluded interface that provides API : getKey()
BufferLedger provides implementation for it.
LowCostIdentityHashMap is not general purpose map it just provides some
“map like” APIs to simplify switch over from IdentityHashMap.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/yufeldman/arrow ARROW-1627-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/arrow/pull/1150.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1150
----
commit be6c350527eda5d8ee582ee183f7a5240bce88f2
Author: Yuliya Feldman <[email protected]>
Date: 2017-09-30T08:14:32Z
ARROW-1627: New class to handle collection of BufferLedger(s) within
AllocationManager
LowCostIdentityHashMap uses array only to store objects since key
(BaseAllocator) is already included in the value (BufferLedger)
To make this class a bit more generic it uses values that implement
ValueWithKeyIncluded interface that provides API : getKey()
BufferLedger provides implementation for it.
LowCostIdentityHashMap is not general purpose map it just provides some
“map like” APIs to simplify switch over from IdentityHashMap.
----
> Reducing memory footprint in AllocationManager.BufferLedger
> -----------------------------------------------------------
>
> Key: ARROW-1627
> URL: https://issues.apache.org/jira/browse/ARROW-1627
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Java - Memory
> Reporter: Yuliya Feldman
> Labels: pull-request-available
>
> Currently we instantiate IdentityHashMap while creating an instance of
> AllocationManager which is in turn is part of BufferLedger
> By default IdentityHashMap allocates an array of 64 objects which is 272
> bytes of heap from the start. In reality a lot of ArrowBuf(s) will not need
> that many entries in the map, since they want share BufferAllocators with
> many other BufferLedgers.
> Suggestion is to come up with different collection to save heap w/o
> sacrificing much of the performance (if any).
> I have a suggestion in mind that I will share via a doc. later today
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
