[INADA Naoki <songofaca...@gmail.com>] > ... > Since current pool size is 4KB and there is pool_header in pool, > we can't allocate 4KB block from pool. > And if support 1KB block, only 3KB of 4KB can be actually used. > I think 512 bytes / 4KB (1/8) is good ratio. > > Do you mean increase pool size? > > How about adding configure option like server-mode? > > SMALL_REQUEST_THRESHOLD 1024 // 2x > POOL_SIZE (16*1024) // 4x > ARENA_SIZE (2*1024*1024) // 8x, and same to huge page size.
While I would like to increase the pool size, it's fraught with danger. The problem: Py_ADDRESS_IN_RANGE has to figure out whether an "arbitrary" address is - or is not - controlled by Python's small-object allocator. The excruciating logic is spelled out at length in obmalloc.c. As is, the code reads up memory near "the start" of a pool to get the pool's belief about which arena it's in. If the memory is not in fact controlled by pymalloc, this can be anything, even uninitialized trash. That's fine (as explained in the comments) - but that memory _must_ be readable. And that's why POOL_SIZE is set to 4K: it's the smallest page size across all the OSes Python is known to run on. If pymalloc is handed an "arbitrary" (but valid) address, the entire OS page containing that address is readable. If, e.g., an OS allocates 4K pages, but Python's POOL_SIZE is 8K (anything bigger than the OS's page allocation unit), then perhaps the OS page at 16K is unallocated, but the page at 20K is. pymalloc sees an address at 21K. As is, pymalloc reads the putative arena index at 20K, which is fine. But if POOL_SIZE were 8K, it would try to read the putative arena index at 16K, and - boom! - segfault. This failure mode would be rare but - of course - catastrophic when it occurs. It would be nice to find a different way for pymalloc to figure out which addresses belong to it. The excruciating Py_ADDRESS_IN_RANGE manages to do it in small constant (independent of the number of arenas and pools in use) time, which is its only virtue ;-) _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
