[
https://issues.apache.org/jira/browse/ARROW-6994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17002984#comment-17002984
]
Wes McKinney commented on ARROW-6994:
-------------------------------------
I tried my test script from ARROW-7305 (requires the file attached there) on
macOS and essentially macOS leaks memory. Really bad
{code}
$ python arrow7305.py
Starting RSS: 67297280
Read CSV RSS: 179224576
Wrote Parquet RSS: 645177344
Waited 1 second RSS: 645177344
Read CSV RSS: 703504384
Wrote Parquet RSS: 707674112
Waited 1 second RSS: 707674112
Read CSV RSS: 760279040
Wrote Parquet RSS: 764452864
Waited 1 second RSS: 764452864
Read CSV RSS: 826171392
Wrote Parquet RSS: 826232832
Waited 1 second RSS: 826232832
Read CSV RSS: 887971840
Wrote Parquet RSS: 892026880
Waited 1 second RSS: 892026880
Read CSV RSS: 942624768
Wrote Parquet RSS: 942669824
Waited 1 second RSS: 942669824
Read CSV RSS: 993071104
Wrote Parquet RSS: 993071104
Waited 1 second RSS: 993071104
Read CSV RSS: 1043841024
Wrote Parquet RSS: 1046388736
Waited 1 second RSS: 1046388736
Read CSV RSS: 1096822784
Wrote Parquet RSS: 1096822784
Waited 1 second RSS: 1096822784
Read CSV RSS: 1147224064
Wrote Parquet RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
Waited 1 second RSS: 1147301888
{code}
I'm seeing about a fix
> [C++] Research jemalloc memory page reclamation configuration on macOS when
> background_thread option is unavailable
> -------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-6994
> URL: https://issues.apache.org/jira/browse/ARROW-6994
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Wes McKinney
> Assignee: Wes McKinney
> Priority: Major
> Fix For: 1.0.0
>
>
> In ARROW-6977, this was disabled on macOS, but this will potentially have
> negative performance and memory implications that were intended to have been
> fixed in ARROW-6910
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
