For the ARENA_SIZE, will that be better to setting by ./configure first, and without hard code in c files?
2017-06-01 17:37 GMT+08:00 INADA Naoki <songofaca...@gmail.com>: > x86's hugepage is 2MB. > And some Linux enables "Transparent Huge Page" feature. > > Maybe, 2MB arena size is better for TLB efficiency. > Especially, for servers having massive memory. > > > On Thu, Jun 1, 2017 at 4:38 PM, Larry Hastings <la...@hastings.org> wrote: >> >> >> When CPython's small block allocator was first merged in late February 2001, >> it allocated memory in gigantic chunks it called "arenas". These arenas >> were a massive 256 KILOBYTES apiece. >> >> This tunable parameter has not been touched in the intervening 16 years. >> Yet CPython's memory consumption continues to grow. By the time a current >> "trunk" build of CPython reaches the REPL prompt it's already allocated 16 >> arenas. >> >> I propose we make the arena size larger. By how much? I asked Victor to >> run some benchmarks with arenas of 1mb, 2mb, and 4mb. The results with 1mb >> and 2mb were mixed, but his benchmarks with a 4mb arena size showed >> measurable (>5%) speedups on ten benchmarks and no slowdowns. >> >> What would be the result of making the arena size 4mb? >> >> CPython could no longer run on a computer where at startup it could not >> allocate at least one 4mb continguous block of memory. >> CPython programs would die slightly sooner in out-of-memory conditions. >> CPython programs would use more memory. How much? Hard to say. It depends >> on their allocation strategy. I suspect most of the time it would be < 3mb >> additional memory. But for pathological allocation strategies the >> difference could be significant. (e.g: lots of allocs, followed by lots of >> frees, but the occasional object lives forever, which means that the arena >> it's in can never be freed. If 1 out of ever 16 256k arenas is kept alive >> this way, and the objects are spaced out precisely such that now it's 1 for >> every 4mb arena, max memory use would be the same but later stable memory >> use would hypothetically be 16x current) >> Many programs would be slightly faster now and then, simply because we call >> malloc() 1/16 as often. >> >> >> What say you? Vote for your favorite color of bikeshed now! >> >> >> /arry >> >> >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/songofacandy%40gmail.com >> > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/me%40louie.lu _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com