[
https://issues.apache.org/jira/browse/ARROW-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16104450#comment-16104450
]
Wes McKinney commented on ARROW-1282:
-------------------------------------
As another thing to try, what happens when you compile jemalloc with a prefix
and statically link? So we would add something like {{je_arrow_}} to all of the
jemalloc functions in
https://github.com/apache/arrow/blob/master/cpp/src/arrow/memory_pool.cc.
Because when libjemalloc is loaded it may interfere with the libc malloc and
free functions, this might make them play more nicely together.
So you would set ARROW_JEMALLOC_USE_SHARED=off in
https://github.com/apache/arrow/blob/master/cpp/CMakeLists.txt#L105 and then
add the prefix in the thirdparty build like in Redis
https://github.com/antirez/redis/blob/unstable/deps/Makefile#L80
> Large memory reallocation by Arrow causes hang in jemalloc
> ----------------------------------------------------------
>
> Key: ARROW-1282
> URL: https://issues.apache.org/jira/browse/ARROW-1282
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++
> Reporter: Jeff Knupp
> Fix For: 0.6.0
>
>
> When reallocating a large amount of memory, Arrow is either triggering a bug
> in jemalloc or has a bug itself in the memory manager (many different
> applications reporting same issue but not clear from jemalloc issue
> description if they're sure it's in jemalloc or caused by other issues like
> using multiple memory allocation libraries in the same process, multithreaded
> access, etc).
> Link to stack trace is here:
> https://gist.github.com/jeffknupp/73879feacf9c560afd4f1a20213dc6ef
> Link to issue in jemalloc GitHub is here:
> https://github.com/jemalloc/jemalloc/issues/802
> Originally observed in redis, discussed with jemalloc maintainer here:
> https://github.com/antirez/redis/issues/3799
> *This is entirely reproducible on Ubuntu 16.04 xenial, which uses version
> 3.6.0 according to `apt` metadata.*
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
