Weston Pace created ARROW-12583:
-----------------------------------
Summary: [Python][C++] pa.total_allocated_bytes incorrect after
switching the default allocator
Key: ARROW-12583
URL: https://issues.apache.org/jira/browse/ARROW-12583
Project: Apache Arrow
Issue Type: Bug
Components: Python
Affects Versions: 4.0.0
Reporter: Weston Pace
For testing I was using a system that had both jemalloc and mimalloc installed.
Given the following script
{code:java}
import pyarrow as pa
import pyarrow.parquet as pq
pa.set_memory_pool(pa.mimalloc_memory_pool())
print(pa.default_memory_pool().backend_name)
table =
pq.read_table('/home/pace/dev/benchmarks/benchmarks/data/temp/fanniemae_2016Q4.uncompressed.parquet')
print(f'A:{table.num_rows}')
print(f'B:{pa.total_allocated_bytes()}')
{code}
The output on Arrow 3.0.0 is:
{code:java}
mimalloc
A:22180168
B:5060201344
{code}
The output on Arrow 4.0.0 is:
{code:java}
mimalloc
A:22180168
B:0
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
