Eddie Elizondo <[email protected]> added the comment:
> the CPU performance implications of adding a branch instruction to Py_INCREC
> and Py_DECREF were, unsurprisingly, quite high.
Yeah, makes sense. I guess it really depends on the specific profile of your
application.
For Instagram this was an overall net positive change and we still have it in
prod. The amount of page faults from Copy on Writes was so large that reducing
it resulted in a net CPU win (even with the added branching). And of course, a
huge reduction in memory usage.
> Microbenchmarks don't tell a good story, the python performance test suite
> should
Agreed. I only added the Richards benchmark as a reference. I'm hoping someone
can pick it up and have more concrete numbers on an average Python workload.
> Given that most people's applications don't fork workers, I do not expect to
> see such an implementation ever become the default.
In any case, I gated this change with ifdefs. In case we don't have it by
default, we can always can easily enable it with a simple
`-DPy_IMMORTAL_INSTANCES` flag to the compiler.
> Also note that this is an ABI change as those INCREF and DECREF definitions
> are intentionally public .h file
This has indeed been a bit of a pain for us as well. Due to how our tooling is
set-up, there's a small number of third party libraries that are still causing
Copy on Writes. Fortunately, that's the only drawback. Even if your
immortalized object goes through an extension that has a different
Py_{DECREF,INCREF} implementation, the reference count will still be so large
that it will never be deallocated.
----------
_______________________________________
Python tracker <[email protected]>
<https://bugs.python.org/issue40255>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
