[Larry Hastings <la...@hastings.org>]
> ...
> Yet CPython's memory consumption continues to grow. By the time a current
> "trunk" build of CPython reaches the REPL prompt it's already allocated 16
> arenas.
I'd be surprised if that's true ;-) The first time `new_arena()` is
called, it allocates space for a vector of 16 (INITIAL_ARENA_OBJECTS)
`arena_object` structs. Those are tiny, and hold bookkeeping info for
the actual arenas, none of which are allocated at first.
`new_arena()` asks the system for memory for just one arena (and 15 of
its 16 initial arena_object structs remain unused, awaiting future
calls).
At the time, 16 wasn't picked because it was the minimum Python
needed, but because it was the _most_ the bulk of a variety of casual
Python scripts needed to complete.
It's certainly gotten worse on a 64-bit box. Here on a 64-bit Win10
under Python 3.6.1 upon reaching the interactive prompt in a DOS box
(set envar PYTHONMALLOCSTATS to get this kind of output whenever
`new_arena()` is called):
# arenas allocated total = 14
# arenas reclaimed = 5
# arenas highwater mark = 9
# arenas allocated current = 9
9 arenas * 262144 bytes/arena = 2,359,296
# bytes in allocated blocks = 2,166,656
# bytes in available blocks = 140,104
0 unused pools * 4096 bytes = 0
# bytes lost to pool headers = 27,648
# bytes lost to quantization = 24,888
# bytes lost to arena alignment = 0
Total = 2,359,296
So at most 9 arenas ("highwater mark") were ever simultaneously allocated..
Change the arena size to 64MB, and most programs would stop with the
first arena allocated ;-)
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
