Quanlong Huang created IMPALA-14064:
---------------------------------------
Summary: Support MemBool backed by BufferPool in parquet scanner
Key: IMPALA-14064
URL: https://issues.apache.org/jira/browse/IMPALA-14064
Project: IMPALA
Issue Type: Improvement
Components: Backend
Reporter: Quanlong Huang
Assignee: Quanlong Huang
TCMalloc is not suitable for allocating/freeing large memory (>256KB)
frequently in multiple threads. The allocation will go into CentralFreeList
that is protected by a global lock. This serializes the threads in large
allocation and free operations.
MemPool is widely used in Backend and currently is backed by TCMalloc.
Operators like scanner could hit this issue so query performance degrades in
high concurrency.
This aims to support MemPool backed by BufferPool which maintains a per-core
arena so allocations will be recycled, alleviating the pressure on the TCMalloc
thread cache. To limit the scope as a first step, only key MemPools used in
parquet scanner will be changed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
