Hello,
This comment is regarding installation with `apt` on ubuntu 18.04 ...
`libarrow-dev/bionic,now 8.0.0-1 amd64`
I'm a bit confused about the memory pool situation:
* I run with `ARROW_DEFAULT_MEMORY_POOL=system` and check that
`arrow::default_memory_pool()->backend_name() ==
arrow::system_memory_pool()->backend_name()`
* I then LD_PRELOAD a customized (*) mimalloc according to the directions
at the mimalloc git repo and things like `strm->Reset(INT32_MAX);` seem not
to be hitting it... I figured that is a big enough chunk to jostle it into
doing something... `BufferOutputStream::Create(INT32_MAX)` is also not
intercepted by mimalloc. Is the "system" pool somehow going around the
typical allocation interfaces on linux? I built my own .so and linked it
to the app and malloc() is getting intercepted.
* `arrow::mimalloc_memory_pool(&mmmp);` does return something... but
apparently not "my" mimalloc ... statically linked?
* what is going on in Arrow with constructor (pre-main()) allocations?
Some of this does hit my LD_PRELOADed mimalloc
* any way to get symbols for the apt-installed libs or would I need to
build from source to get backtrace with symbols? (for chasing down sources
of allocations)
* what is the C++ lib equivalent of the following from the Python code? I
figure I could stop trying to understand the built-in/default allocators if
I could just replace them... but this may also intersect with my question
about constructors. Maybe I'd have to make sure my constructor runs first
to perform the switch-a-roo before anything else tries to use the default
pool?
```
namespace py {
static std::mutex memory_pool_mutex;
static MemoryPool* default_python_pool = nullptr;
void set_default_memory_pool(MemoryPool* pool) {
std::lock_guard<std::mutex> guard(memory_pool_mutex);
default_python_pool = pool;
}
```
(*) the mimalloc customization: the main app has a weak reference that ends
up defined by the LD_PRELOAD mimalloc, where the function so-supplied
allows the app to install a function pointer (back to the main app) that
gets called (if defined) at various interesting points in mimalloc
Thanks,
John
