> This is a relatively‑basic and easily‑triggered issue. Should we document it
> in the official documentation?
If glibc malloc is used by default, it is recommended to limit
MALLOC_ARENA_MAX. For example, add the following line in start‑worker.sh:
> export MALLOC_ARENA_MAX=${MALLOC_ARENA_MAX:-4}
Alternatively, we could recommend using Jemalloc to prevent excessive memory
fragmentation.
> 2026年8月25日 15:02，Cheng Pan <[email protected]> 写道：
>
> I would recommend to use jemalloc instead.
>
> Thanks,
> Cheng Pan
>
>
>
>> On Aug 25, 2026, at 12:05, 毛玉伟 <[email protected]> wrote:
>>
>> Hi community,
>>
>> In our Celeborn cluster, the MALLOC_ARENA_MAX environment variable is not
>> configured on the Worker machines. Without this configuration, glibc
>> defaults to 8 * number of CPU cores. Since our machines have 128 cores,
>> MALLOC_ARENA_MAX defaults to $8 \times 128 = 1024$.
>>
>> Under this setup, the memory occupied by glibc arenas in the Worker process
>> is extremely high. For example, arena=7.58GiB, heaps=545, and
>> totalRSS=23.17GiB. Furthermore, the memory consumed by these arenas is not
>> accounted for in Direct memory usage by the MemoryManager, leading to an
>> excessively high overall memory footprint for the Worker process.
>>
>>
>>
>> Question：
>>
>> Our research indicates that many big data components recommend setting
>> MALLOC_ARENA_MAX to 4 or 8. Does the Celeborn community have a recommended
>> value for this? Additionally, are there any future plans to address the
>> issue where Direct memory usage does not account for glibc arena memory?
>
