Jokser opened a new issue, #13837:
URL: https://github.com/apache/arrow/issues/13837
Hi, everybody,
Recently, I noticed that my Arrow Flight Server keeps a lot of resident
memory (hundred of gigabytes) after ingestion of record batches via `DoPut`
call.
After debugging I found out that this memory is used in arrow memory pools.
I tried to get memory stats from default and system pools, but it seems that
stats are broken:
```
spdlog::info("[Default] Memory usage: peak = {},
allocated = {}", arrow::default_memory_pool()->max_memory(),
arrow::default_memory_pool()->bytes_allocated());
spdlog::info("[System] Memory usage: peak = {},
allocated = {}", arrow::system_memory_pool()->max_memory(),
arrow::system_memory_pool()->bytes_allocated());
```
The output of these commands always shows 0 for peak and allocated.
However, when I manually call
`arrow::default_memory_pool()->ReleaseUnused();` resident memory of the process
is freed.
So I have 2 questions
1. Is there any way to control the memory usage of memory pools other than
periodically explicitly releasing pools after some idle time?
2. Is there any issue with broken memory pool stats? I didn't find anything
related to it in jira.
Thank you in advance.
Arrow version: 6.0.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
