[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Eric Snow added the comment: Thanks for the updates, Eddie. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Eddie Elizondo added the comment: It seems that we are back on track with perf being back to neutral! I've created 4 new PRs. Each with an optimization applied on top of the baseline introduction of instance immortalization. The main PR 19474 currently stands at around 4%, after rebasing pas

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Inada Naoki added the comment: All of these optimizations should be disabled by default. * It will cause leak when Python is embedded. * Even for python command, it will break __del__ and weakref callbacks. -- ___ Python tracker

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Change by Eddie Elizondo : -- pull_requests: +29621 pull_request: https://github.com/python/cpython/pull/31491 ___ Python tracker ___ __

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Change by Eddie Elizondo : -- pull_requests: +29620 pull_request: https://github.com/python/cpython/pull/31490 ___ Python tracker ___ __

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Change by Eddie Elizondo : -- pull_requests: +29619 pull_request: https://github.com/python/cpython/pull/31489 ___ Python tracker ___ __

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Change by Eddie Elizondo : -- pull_requests: +29618 pull_request: https://github.com/python/cpython/pull/31488 ___ Python tracker ___ __

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Inada Naoki added the comment: I think making more objects immortal by default will reduce the gap, although I am not sure it can be 2%. (I guess 3% and I think it is acceptable gap.) * Code attributes (contents of co_consts, co_names, etc...) in deep frozen modules. * only if subinterpret

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Eddie Elizondo added the comment: @eric.snow great to hear about this update! I'll start looking at some of the techniques that we talked about to improve performance, I'm optimistic that we'll be able to close down the gap to 2%. -- ___ Python tr

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Eric Snow added the comment: @Eddie, what can I do to push this forward? FYI, in addition to the python-dev thread a few weeks back, I've brought the matter up with the steering council. [1] Also, if we can get back to performance-neutral (currently at about 4% slower) then there would be

[issue40255] Fixing Copy on Writes from reference counting and immortal objects

Change by STINNER Victor : -- title: Fixing Copy on Writes from reference counting -> Fixing Copy on Writes from reference counting and immortal objects ___ Python tracker ___

[issue40255] Fixing Copy on Writes from reference counting

STINNER Victor added the comment: I just want to say that GOOGLE_ETERNAL_REFCOUNT_SUPPORT is a cool name :-D I love "eternal refcount"! -- ___ Python tracker ___

[issue40255] Fixing Copy on Writes from reference counting

Gregory P. Smith added the comment: [data] I finally dug up the old YouTube doc at work with their findings. Mostly just posting this here for future public reference if anyone wants. Nothing surprising. When youtube experimented with a modified 2.7 adding "eternal refcounts" in 2015, they

[issue40255] Fixing Copy on Writes from reference counting

Eric Snow added the comment: While the eventual solution may overlap, my interests are divergent enough from those here that I've created a separate issue: bpo-43503. -- ___ Python tracker _

[issue40255] Fixing Copy on Writes from reference counting

Eric Snow added the comment: Note that my PR does make the builtin types immortal (at least those initialized in _PyTypes_Init()). However, I did not make the singletons or other candidate objects immortal in the PR. That can be done separately. --

[issue40255] Fixing Copy on Writes from reference counting

Eric Snow added the comment: FYI, I've just put up a similar PR [1] to Eddie's, with a focus on object immortality rather than immutability. However, if Py_IMMORTAL_CONST_REFCOUNTS is defined then it should have the behavior Eddie is after. [1] https://github.com/python/cpython/pull/24828

[issue40255] Fixing Copy on Writes from reference counting

Change by Eric Snow : -- nosy: +eric.snow nosy_count: 15.0 -> 16.0 pull_requests: +23592 pull_request: https://github.com/python/cpython/pull/24828 ___ Python tracker ___ _

[issue40255] Fixing Copy on Writes from reference counting

Inada Naoki added the comment: >> Fast shutdown option > > You can use os._exit(0). Yes. Instagram use it as `atexit.register(os._exit, 0)`. https://instagram-engineering.com/dismissing-python-garbage-collection-at-instagram-4dca40b29172 I think this hack can be supported in multiprocessing

[issue40255] Fixing Copy on Writes from reference counting

STINNER Victor added the comment: > Fast shutdown option You can use os._exit(0). -- ___ Python tracker ___ ___ Python-bugs-list m

[issue40255] Fixing Copy on Writes from reference counting

Change by Inada Naoki : -- versions: +Python 3.10 -Python 3.9 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40255] Fixing Copy on Writes from reference counting

Inada Naoki added the comment: I'm big -1 too. But I am interested in Instagram usage. * How % of heap are CoW-ed with gc.freeze()? * When CoW happen? in execution time, or shutdown? * Which type cause CoW? I have two ideas to reduce CoW: * Fast shutdown option Currently Python try to free

[issue40255] Fixing Copy on Writes from reference counting

Mark Shannon added the comment: On 20/04/2020 2:33 pm, Steve Dower wrote: > > Steve Dower added the comment: > >> I would expect that the negative impact on branch predictability would >> easily outweigh the cost of the memory write (A guaranteed L1 hit) > > If that were true then Spectre

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: Wow. I see, thank you. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue40255] Fixing Copy on Writes from reference counting

Rémi Lapeyre added the comment: >> We also have the real world app that is Instagram. > I don't think Instagram is a single app. What is Python used for at Instagram? According to their engineering blog (https://instagram-engineering.com/static-analysis-at-scale-an-instagram-story-8f498ab71a0

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: > We also have the real world app that is Instagram. I don't think Instagram is a single app. What is Python used for at Instagram? -- ___ Python tracker __

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: > But micro-benchmarks may tell you things which are not true in the real-world > (for example an easily-predicted branch) This is true, though this branch should be more easily predictable because INCREF/DECREF are inlined. If there were only one function doin

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: > Benchmarking already showed that the branching version is faster. But micro-benchmarks may tell you things which are not true in the real-world (for example an easily-predicted branch). -- ___ Python tracker

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: > I would expect that the negative impact on branch predictability would easily > outweigh the cost of the memory write (A guaranteed L1 hit) If that were true then Spectre and Meltdown wouldn't have been so interesting :) Pipelining processors are going to spe

[issue40255] Fixing Copy on Writes from reference counting

Mark Shannon added the comment: Eddie, How did you compare the branching vs. non-branching implementations? I have read the code in the PR. What is the non-branching version that you used to compare it against? Why not use the sign bit as the immortal bit? It simplifies the test considerab

[issue40255] Fixing Copy on Writes from reference counting

Neil Schemenauer added the comment: I resurrected an old thread on Discourse that seems quite relevant to this PR: https://discuss.python.org/t/switching-from-refcounting-to-libgc/1641/35?u=nas -- ___ Python tracker

[issue40255] Fixing Copy on Writes from reference counting

Neil Schemenauer added the comment: > immutability vs immutable object headers Sorry, I meant to write "immortal vs immutable". -- ___ Python tracker ___

[issue40255] Fixing Copy on Writes from reference counting

Neil Schemenauer added the comment: A few thoughts First, the PR is well done. The changes are pretty minimal and are well localized. Thanks Eddie for writing clean code and responding to review requests. My feeling is that, in current state, this still should not get merged or at least

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: > which would potentially save the two word per object overhead Btw, I misread. I thought `gc_bits` referred to the bits used by the GC in the reference count. In any case, you can still use a bit in the reference count :) -- ___

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: > I'm somewhat puzzled how a version that does no more work and has no jumps is > slower. Oh I think I know why there's some confusion. I've updated the PR from the initial version (which is probably the one that you saw). The branching does less work in Py

[issue40255] Fixing Copy on Writes from reference counting

Mark Shannon added the comment: Do you have more details on your comparison? I'm somewhat puzzled how a version that does no more work and has no jumps is slower. The memory hit is not immediate, but making the object header immutable prevents changes like https://github.com/markshannon/cpy

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: Mark: > You are asking the whole world to take a hit on both performance and memory > use. That's an explicit non-goal. That's why the code was guarded to add as an optional compilation mode This should be added by default if and only if this is a neutral o

[issue40255] Fixing Copy on Writes from reference counting

Mark Shannon added the comment: A big -1 to this. You are asking the whole world to take a hit on both performance and memory use, in order to save Instagram memory. The PR uses the term "immortal" everywhere. There is only one reference to copy-on-write in a comment. Yet this issue about m

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: After the added optimizations and assuming that running this more rigorously (i.e PGO + LTO + full cpu isolation + speed.python.org machine) also shows the PR as perf neutral, what are people's thoughts so far? Would this be enough to consider adding this ch

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: Neil: > The fastest would be to create an immortal block as part of the BSS > (uninitialized data). That's an interesting idea, definitely worth exploring and we can probably get some perf win out of it. And yes, using the frozen modules is definitely a step

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: I was able to get an equal (maybe slightly better) performance by following the advice from Steve and Neil and skipping reference counting of instances that we know are immortal. It seems that by getting performance parity, we should be able to ease most of

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: Makes sense. Yes, caution is required about what code runs before fork, but forkserver’s solution for that would be a non-starter for us, since it would ensure that we can share no basically no memory at all between worker processes. -- _

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: Steve is right. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https:/

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: > What problem do you have in mind that the fork server would solve? My understanding is that the fork server solves the *general* problem of running arbitrary code before fork by making sure that only CPython/multiprocessing gets to run code before fork. In t

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: > I would be interested to hear the answer to Antoine's question which is > basically: why not using the multiprocessing fork server? Concretely, because for a long time we have used the uWSGI application server and it manages forking worker processes (among oth

[issue40255] Fixing Copy on Writes from reference counting

Neil Schemenauer added the comment: Eddie mentions in the PR about using memory arenas to contain immortal objects. I think it could be worth investigating further. With the current PR, the immortal status is dependent on the value of the refcnt field of the object. Using immortal arenas

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: I'll note that this "extremely common" model can break as soon as you have hidden worker threads somewhere (this can happen in a third-party library). For example, the libhdfs library is not fork-safe. -- ___ Pyth

[issue40255] Fixing Copy on Writes from reference counting

STINNER Victor added the comment: Carl: > A lot of the "big data" in question here is simply lots of Python > module/class/code objects resulting from importing lots of Python modules. > > And yes, this "pre-fork" model is extremely common for serving Python web > applications; it is the way

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: > Is it a common use case to load big data and then fork to use preloaded data? A lot of the "big data" in question here is simply lots of Python module/class/code objects resulting from importing lots of Python modules. And yes, this "pre-fork" model is extreme

[issue40255] Fixing Copy on Writes from reference counting

STINNER Victor added the comment: This feature seems to be driven by Instagram use case. Are there other users who would benefit of that? Is it a common use case to load big data and then fork to use preloaded data? PR 19474 has a maintenance cost: * new configuration option * new macro * n

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > I'm not sure what you mean here by "balanced ref count" or by "work" :) What > will happen anytime an immortal object gets into the GC, for any reason, is > that the GC will "subtract" cyclic references and see that the immortal > object still has a

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: I think the concerns about "perfect" behavior in corner cases are in general irrelevant here. In the scenarios where this optimization matters, there is no quantitative change that occurs at 100% coverage. Preventing 99% of CoW is 99% as good as preventing 100%

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > But if said objects (isolated and untracked before and now tracked) acquire > strong references to immortal objects, those objects will be visited when the > gc starts calculating the isolated cycles and that requires a balanced > reference count to

[issue40255] Fixing Copy on Writes from reference counting

Gregory P. Smith added the comment: Marking everything as immortal/eternal after you've loaded your common code & data but before you do your forking is thenorm for this kind of serving system. You already presumably have a defined lifetime for the processes so they'll likely be set to die w

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > all objects that are tracked by the gc also, all the objects that are reachable from objects tracked by the gc. -- ___ Python tracker _

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > There hasn't been much said about which kind of objects would be immortal. The current patch makes *all objects that are tracked by the gc* immortal, independently of their type (unless I am missing something). -- __

[issue40255] Fixing Copy on Writes from reference counting

Change by Steve Dower : -- nosy: +dino.viehland ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: There hasn't been much said about which kind of objects would be immortal. Could this be a creation-time option? Or a type object (similar to a no-op tp_dealloc)? If it's something that is known when the object is created, then some of the "infection" concern

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > I think this is simply expected behavior if you choose to create immortal > objects, and not really an issue. How could you have an immortal object that > doesn't keep its strong references alive? I think I didn't express myself very well: I am sayi

[issue40255] Fixing Copy on Writes from reference counting

Dino Viehland added the comment: I think there's other cases of performance related features being hidden under an ifdef. Computed gotos show up that way, although probably more because it's a compiler extension that's not supported everywhere. Pymalloc is also very similar in that it impl

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: > This may break the garbage collector algorithm that relies on the balance > between strong references between objects and its reference count to do the > calculation of the isolated cycles. I don't think it really breaks anything. What happens is that the immo

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: > An immortalized object will never start participating in reference counting > again after it is immortalized. Well, "passed to an extension compiled with no-immortal headers" is an exception to this. But for the "not GC tracked but later becomes GC tracked" c

[issue40255] Fixing Copy on Writes from reference counting

Carl Meyer added the comment: > Anything that is touched by the immortal object will be leaked. This can also > happen in obscure ways if reference cycles are created. I think this is simply expected behavior if you choose to create immortal objects, and not really an issue. How could you ha

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: As a separate discussion, I would be interested to know whether the original use case (Eddie's) could be satisfied differently. It probably doesn't belong to this issue, though. (Eddie, if you want to discuss this, feel free to e-mail me privately. I think

[issue40255] Fixing Copy on Writes from reference counting

Antoine Pitrou added the comment: > I run the pyperformance test suite with PGO + LTO + full cpu isolation in the > speed.python.org machine and these were the results: Be mindful that synthetic benchmarks are probably a gentle case for branch prediction. Real-world performance on irregular

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: >> This isn't actually about removing immortal objects, but removing *mutable* >> objects that would be shared between subinterpreters. Making some objects >> immortal, such as interned strings or stateless singletons, would actually >> benefit this work, as they

[issue40255] Fixing Copy on Writes from reference counting

Change by hai shi : -- nosy: +shihai1991 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.

[issue40255] Fixing Copy on Writes from reference counting

STINNER Victor added the comment: I would be more interested by an experiment to move ob_refcnt outside PyObject to solve the Copy-on-Write issue, especially to measure the impact on performances. -- ___ Python tracker

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > gc.freeze() has a similar issue, no? This function also comes from Instagram > specific use case ;-) Not really because this patch is much more impacting. gc.freeze() moves objects into a permanent generation making them not participate in the GC so

[issue40255] Fixing Copy on Writes from reference counting

STINNER Victor added the comment: The idea of immortal objects was proposed to solve the bpo-39511 problem. I dislike immortable for different reasons. Here are some. Gregory: > We wound up not deploying it or pushing for it in CPython because the CPU > performance implications of adding a b

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: > There is a trend right now to try to remove immortal objects (started by > Victor and others) like static types and transforming them to heap types. This isn't actually about removing immortal objects, but removing *mutable* objects that would be shared betwe

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: > If it's easy to set up, it might be interesting to see if there's a > difference if (e.g.) all interned strings are marked as immortal, or all > small ints. Or type objects. Maybe set a flag during initialization and treat > everything created befor

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: Thanks, Eddie for sharing the patch. I think this is an interesting discussion: for instance, in the core dev sprint at Microsoft I recall that we had a quick discussion about having immortal references. After analyzing the patch, and even assuming t

[issue40255] Fixing Copy on Writes from reference counting

Steve Dower added the comment: If it's easy to set up, it might be interesting to see if there's a difference if (e.g.) all interned strings are marked as immortal, or all small ints. Or type objects. Maybe set a flag during initialization and treat everything created before that as immortal

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: System state CPU: use 12 logical CPUs: 1,3,5,7,9,11,13,15,17,19,21,23 Perf event: Maximum sample rate: 1 per second ASLR: Full randomization Linux scheduler: Isolated CPUs (12/24): 1,3,5,7,9,11,13,15,17,19,21,23 Linux scheduler: RCU disable

[issue40255] Fixing Copy on Writes from reference counting

Pablo Galindo Salgado added the comment: I run the pyperformance test suite with PGO + LTO + full cpu isolation in the speed.python.org machine and these were the results: +-+--+---+ | Benchmark

[issue40255] Fixing Copy on Writes from reference counting

Change by Steve Dower : -- nosy: +steve.dower ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.py

[issue40255] Fixing Copy on Writes from reference counting

Change by Dong-hee Na : -- nosy: +corona10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pytho

[issue40255] Fixing Copy on Writes from reference counting

Eddie Elizondo added the comment: > the CPU performance implications of adding a branch instruction to Py_INCREC > and Py_DECREF were, unsurprisingly, quite high. Yeah, makes sense. I guess it really depends on the specific profile of your application. For Instagram this was an overall net

[issue40255] Fixing Copy on Writes from reference counting

Gregory P. Smith added the comment: Thanks for sharing! It's good to have a patch implementing for others who might need it to try out. We experimented with an implementation of this on CPython 2.7 that we called Eternal Refcounts for YouTube many years ago. For the same reason (saving me

[issue40255] Fixing Copy on Writes from reference counting

Change by Dong-hee Na : -- nosy: +pablogsal ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.pyth

[issue40255] Fixing Copy on Writes from reference counting

Change by Pablo Galindo Salgado : -- nosy: +nascheme, pitrou, tim.peters ___ Python tracker ___ ___ Python-bugs-list mailing list Un

[issue40255] Fixing Copy on Writes from reference counting

Change by Eddie Elizondo : -- keywords: +patch pull_requests: +18829 stage: -> patch review pull_request: https://github.com/python/cpython/pull/19474 ___ Python tracker ___ _

[issue40255] Fixing Copy on Writes from reference counting

New submission from Eddie Elizondo : Copy on writes are a big problem in large Python application that rely on multiple processes sharing the same memory. With the implementation of `gc.freeze`, we've attenuated the problem by removing the CoW coming from the GC Head. However, reference count