On Sat, 15 Jun 2019 01:15:11 -0500 Tim Peters <tim.pet...@gmail.com> wrote: > > > ... > > My feeling right now is that Tim's obmalloc-big-pool is the best > > design at this point. Using 8 KB or 16 KB pools seems to be better > > than 4 KB. The extra complexity added by Tim's change is not so > > nice. obmalloc is already extremely subtle and obmalloc-big-pool > > makes it more so. > > Moving to bigger pools and bigger arenas are pretty much no-brainers > for us, [...]
Why "no-brainers"? Bigger pools sound ok, but bigger arenas will make Python less likely to return memory to the system. We should evaluate what problem we are trying to solve here, instead of staring at micro-benchmark numbers on an idle system. Micro-benchmarks don't tell you what happens on a loaded system with many processes, lots of I/O happening. If the problem is the cost of mmap() and munmap() calls, then the solution more or less exists at the system level: jemalloc and other allocators use madvise() with MADV_FREE (see here: https://lwn.net/Articles/593564/). A possible design is a free list of arenas on which you use MADV_FREE to let the system know the memory *can* be reclaimed. When the free list overflows, call munmap() on extraneous arenas. Regards Antoine. _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MJC274FJJMAD3C25XV5S6UOKNPAZ5A3R/
