On Wed, Feb 23, 2022 at 1:46 AM Eddie Elizondo via Python-Dev <python-dev@python.org> wrote: > > > That article is five years old so it doesn't reflect the current state of the > system! We have continuous profiling and monitoring of Copy on Writes and > after introducing the techniques described in this PEP, we have largely fixed > the majority of scenarios where this happens. > > You are right in the fact that just addressing reference counting will not > fix all CoW issues. The trick here is also to leverage the permanent GC > generation used for the `gc.freeze` API. That is, if you have a container > that it's known to be immortal, it should be pushed into the permanent GC > generation. This will guarantee that the GC itself will not change the GC > headers of said instance. > > Thus, if you immortalize your heap before forking (using the techniques in: > https://github.com/python/cpython/pull/31489) then you'll end up removing the > vast majority of scenarios where CoW takes place. I can look into writing a > new technical article for Instagram with more up to date info but this might > take time to get through! > > Now, I said that we've largely fixed the CoW issue because there are still > places where it happens such as: free lists, the small object allocator, etc. > But these are relatively small compared to the ones coming from reference > counts and the GC head mutations.
Same technique don't guarantee same benefit. Like gc.freeze() is needed before immortalize to avoid CoW, some other tricks may be needed too. New article is welcome, but I want sample application we can run, profile, and measure the benefits. Regards, -- Inada Naoki <songofaca...@gmail.com> _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AUF5R62E7YT22LL4DJ5HI3FCS3ZPHSTL/ Code of Conduct: http://python.org/psf/codeofconduct/
