[
https://issues.apache.org/jira/browse/ARROW-1760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235927#comment-16235927
]
ASF GitHub Bot commented on ARROW-1760:
---------------------------------------
jacques-n commented on issue #36: ARROW-1760: [Java] Add Apache Mnemonic
(incubating) as alternative backed allocator
URL: https://github.com/apache/arrow/pull/36#issuecomment-341459359
I think we should look at doing this in a cleaner way. Having setters on
static interface seems like a bit of hack. I also think it probably makes sense
to expose a location property (or similar) as well as an ability to move memory
between domains. A good way might be to have an optional constructor for
RootAllocator with a new interface. The default could have a wrapped version of
the existing static pooled udle allocator.
The allocator capacity should also be tied to the subsystem. Right now I
think we're constrained by directory memory capacity of the JVM but that may
not be true in the case that we're using the other allocator.
Also, any idea on the performance using the alternative allocator. Does
Mnemonic have it's own intelligent allocator? The normal path uses a nice
allocator to manage various size chunks. The model presented here operates
above that allocator (it seems like maybe it should be below the netty
allocator and used for chunk allocations rather than final allocations) and
thus I wonder how smaller allocations would work (I don't know Mnemonic well).
It seems like people should be able to inspect as well change the memory
tier that a ArrowBuf can be located in. For example, move a buffer between
memory, nvme and disk. Thoughts? In this case we need think more about how we
manage allocation tracking. You'd potentially want have constraints and/or
reservations per domain.
I also would prefer not making mnemonic a required dependency. Seems like we
should look at how we can make it optional. If we do something more interface
based at the RootAllocator level, this should be possible.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> [Java] Add Apache Mnemonic (incubating) as alternative allocator for
> storage-class memory support
> -------------------------------------------------------------------------------------------------
>
> Key: ARROW-1760
> URL: https://issues.apache.org/jira/browse/ARROW-1760
> Project: Apache Arrow
> Issue Type: New Feature
> Components: Java - Memory
> Reporter: Wes McKinney
> Labels: pull-request-available
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
