[
https://issues.apache.org/jira/browse/ARROW-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16912776#comment-16912776
]
Wes McKinney commented on ARROW-4120:
-------------------------------------
In the context of ARROW-6060, we had a case with runaway peak memory use.
Otherwise there was no memory leak nor issue with Python references etc.
I think we should implement tests that check that peak memory use (at least
according to what a {{MemoryPool}} is able to account for) does not exceed a
given level. Then we could write better tests
> [Python] Define process for testing procedures that check for no macro-level
> memory leaks
> -----------------------------------------------------------------------------------------
>
> Key: ARROW-4120
> URL: https://issues.apache.org/jira/browse/ARROW-4120
> Project: Apache Arrow
> Issue Type: Improvement
> Components: Python
> Reporter: Wes McKinney
> Priority: Major
> Fix For: 1.0.0
>
>
> Some kinds of memory leaks may be difficult to unit test for, and they may
> not cause valgrind errors necessarily
> I had written some ad hoc leak tests in
> https://github.com/apache/arrow/blob/master/python/scripts/test_leak.py. We
> have some more of this in ARROW-3324.
> It would be useful to be able to create a sort of "test suite" of memory leak
> checks. They are a bit too intensive to run in CI (since you may have to run
> something many iterations to see whether it leaks), but we could run them in
> a nightly build
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
