It seems very complex and not portable at all to "free" a part of an arena. We already support freeing a whole arena using munmap(). I was a huge enhancement in our memory allocator. Change made in Python 2.5? I don't recall, ask Evan Jones: http://www.evanjones.ca/memoryallocator/ :-)
I'm not sure that it's worth it to increase the arena size and try to implement the MADV_DONTNEED / MADV_FREE thing. Victor 2017-06-01 11:21 GMT+02:00 INADA Naoki <songofaca...@gmail.com>: > Thanks for detailed info. > > But I don't think it's a big problem. > Arenas are returned to system by chance. So other processes > shouldn't relying to it. > > And I don't propose to stop returning arena to system. > I just mean per pool (part of arena) MADV_DONTNEED can reduce RSS. > > If we use very large arena, or stop returning arena to system, > it can be problem. > > Regards, > > On Thu, Jun 1, 2017 at 6:05 PM, Siddhesh Poyarekar <siddh...@gotplt.org> > wrote: >> On Thursday 01 June 2017 01:53 PM, INADA Naoki wrote: >>> * On Linux, madvice(..., MADV_DONTNEED) can be used. >> >> madvise does not reduce the commit charge in the Linux kernel, so in >> high consumption scenarios (and where memory overcommit is disabled or >> throttled) you'll see programs dying with OOM despite the MADV_DONTNEED. >> The way we solved it in glibc was to use mprotect to drop PROT_READ and >> PROT_WRITE in blocks that we don't need when we detect that the system >> is not configured to overcommit (using /proc/sys/vm/overcommit_memory). >> You'll need to fix the protection again though if you want to reuse the >> block. >> >> Siddhesh > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
