[Python-Dev] Re: radix tree arena map for obmalloc

2019-06-14 Thread Tim Peters
[Neil Schemenauer ] > ... > BTW, the current radix tree doesn't even require that pools are > aligned to POOL_SIZE. We probably want to keep pools aligned > because other parts of obmalloc rely on that. obmalloc relies on it heavily. Another radix tree could map block addresses to all the necess

[Python-Dev] Re: radix tree arena map for obmalloc

2019-06-14 Thread Neil Schemenauer
Here are benchmark results for 64 MB arenas and 16 kB pools. I ran without the --fast option and on a Linux machine in single user mode. The "base" columm is the obmalloc-big-pools branch with ARENA_SIZE = 64 MB and POOL_SIZE = 16 kB. The "radix" column is obmalloc_radix_tree (commit 5e00f6041)

[Python-Dev] Re: radix tree arena map for obmalloc

2019-06-14 Thread Neil Schemenauer
On 2019-06-14, Tim Peters wrote: > However, last I looked there Neil was still using 4 KiB obmalloc > pools, all page-aligned. But using much larger arenas (16 MiB, 16 > times bigger than my branch, and 64 times bigger than Python currently > uses). I was testing it verses your obmalloc-big-pool

[Python-Dev] Re: radix tree arena map for obmalloc

2019-06-14 Thread Neil Schemenauer
On 2019-06-15, Inada Naoki wrote: > Oh, do you mean your branch doesn't have headers in each page? That's right. Each pool still has a header but pools can be larger than the page size. Tim's obmalloc-big-pool idea writes something to the head of each page within a pool. The radix tree doesn't

[Python-Dev] Re: Who uses libpython38.a on Windows?

2019-06-14 Thread MRAB
On 2019-06-14 21:53, Steve Dower wrote: One of the most annoying steps in building the Windows installers is generating the libpython38.a file. It's annoying, because it requires having "generic enough" MinGW tools to ensure that the file is compatible with whatever version of MinGW might be tryi

[Python-Dev] Re: radix tree arena map for obmalloc

2019-06-14 Thread Tim Peters
[Inada Naoki . to Neil S] > Oh, do you mean your branch doesn't have headers in each page? That's probably right ;-) Neil is using a new data structure, a radix tree implementing a sparse set of arena addresses. Within obmalloc pools, which can be of any multiple-of-4KiB (on a 64-bit box) size,

[Python-Dev] Re: radix tree arena map for obmalloc

2019-06-14 Thread Inada Naoki
Oh, do you mean your branch doesn't have headers in each page? https://bugs.python.org/issue32846 As far as I remember, this bug was caused by cache thrashing (page header is aligned by 4K, so cache line can conflict often.) Or this bug can be caused by O(N) free() which is fixed already. I'll s

[Python-Dev] Who uses libpython38.a on Windows?

2019-06-14 Thread Steve Dower
One of the most annoying steps in building the Windows installers is generating the libpython38.a file. It's annoying, because it requires having "generic enough" MinGW tools to ensure that the file is compatible with whatever version of MinGW might be trying to build against the regular Window

[Python-Dev] radix tree arena map for obmalloc

2019-06-14 Thread Neil Schemenauer
I've been working on this idea for a couple of days. Tim Peters has being helping me out and I think it has come far enough to get some more feedback. It is not yet a good replacement for the current address_in_range() test. However, performance wise, it is very close. Tim figures we are not do

[Python-Dev] Summary of Python tracker Issues

2019-06-14 Thread Python tracker
ACTIVITY SUMMARY (2019-06-07 - 2019-06-14) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open7022 (+15) closed 42013 (+70) total 49035 (+85) Open issues wi

[Python-Dev] Re: PyAPI_FUNC() is needed to private APIs?

2019-06-14 Thread Jeroen Demeyer
On 2019-06-13 18:03, Inada Naoki wrote: We don't provide method calling API which uses optimization same to LOAD_METHOD. Which may be like this: /* methname is Unicode, nargs > 0, and args[0] is self. */ PyObject_VectorCallMethod(PyObject *methname, PyObject **args, Py_ssize_t nargs, PyObject