[
https://issues.apache.org/jira/browse/ARROW-12584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17334711#comment-17334711
]
David Li commented on ARROW-12584:
----------------------------------
jemalloc isn't under question here, but it can have similar behaviors and
offers a similar call to drop all unused memory, if we ever run into issues.
{noformat}
arena.<i>.decay (void) --
Trigger decay-based purging of unused dirty/muzzy pages for arena <i>, or
for all arenas if <i> equals MALLCTL_ARENAS_ALL. The proportion of unused
dirty/muzzy pages to be purged depends on the current time; see
opt.dirty_decay_ms and opt.muzy_decay_ms for details.
arena.<i>.purge (void) --
Purge all unused dirty pages for arena <i>, or for all arenas if <i> equals
MALLCTL_ARENAS_ALL.
{noformat}
> [C++][Python] Expose method for benchmarking tools to release unused memory
> from the allocators
> -----------------------------------------------------------------------------------------------
>
> Key: ARROW-12584
> URL: https://issues.apache.org/jira/browse/ARROW-12584
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Python
> Reporter: Weston Pace
> Assignee: Weston Pace
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> A team working on Arrow benchmarks encountered a problem where the mimalloc
> allocator was holding onto considerably more RAM than needed (at least 6GB
> and potentially more).
> Mimalloc offers the `mi_collect` non-standard API which forces mimalloc to
> release unused pages to the OS. We could expose this to be called by
> benchmarking tools to get consistent results and reduce the amount of RAM
> needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
