[issue42197] Disable automatic update of frame locals during tracing

STINNER Victor added the comment: Mark: Please add the new PyFrame_GetLocals() function to the C API > New Features doc: https://docs.python.org/dev/whatsnew/3.11.html#id1 -- ___ Python tracker

[issue42197] Disable automatic update of frame locals during tracing

Mark Shannon added the comment: New changeset d7163bb35d1ed46bde9affcd4eb267dfd0b703dd by Mark Shannon in branch 'main': bpo-42197: Don't create `f_locals` dictionary unless we actually need it. (GH-32055) https://github.com/python/cpython/commit/d7163bb35d1ed46bde9affcd4eb267dfd0b703dd

[issue42197] Disable automatic update of frame locals during tracing

STINNER Victor added the comment: See also bpo-47092: [C API] Add PyFrame_GetVar(frame, name) function. -- ___ Python tracker ___

[issue42197] Disable automatic update of frame locals during tracing

Change by Mark Shannon : -- pull_requests: +30147 pull_request: https://github.com/python/cpython/pull/32055 ___ Python tracker ___

[issue42197] Disable automatic update of frame locals during tracing

Ned Batchelder added the comment: And btw, the tests with that branch ran at least twice as fast as with 3.10! -- ___ Python tracker ___

[issue42197] Disable automatic update of frame locals during tracing

Ned Batchelder added the comment: I ran the coverage.py test suite using https://github.com/faster-cpython/cpython/tree/dont-fast-to-locals-in-trampoline, and there were no failures. -- ___ Python tracker

[issue42197] Disable automatic update of frame locals during tracing

STINNER Victor added the comment: As I already wrote previously, I'm a supporter of requiring debuggers and profilers to explicitly call PyFrame_FastToLocalsWithError() and PyFrame_LocalsToFast(). It should only impact a minority of users, whereas the majority will benefit of way better

[issue42197] Disable automatic update of frame locals during tracing

Mark Shannon added the comment: Or you can use this branch: https://github.com/faster-cpython/cpython/tree/dont-fast-to-locals-in-trampoline -- ___ Python tracker ___

[issue42197] Disable automatic update of frame locals during tracing

Mark Shannon added the comment: Yes the PR has all the changes. It is just the changes sysmodule.c that you need: https://github.com/python/cpython/pull/23028/files#diff-a3a5c73931235f7f344c072dc755d6508e13923db3f5d581c5e88652075871cb -- ___

[issue42197] Disable automatic update of frame locals during tracing

Ned Batchelder added the comment: Off the top of my head, I'm not sure this affects coverage.py at all, but I could be missing something. Does PR 23028 have all the changes, or is there some other python/cpython branch I can test with? -- ___

[issue42197] Disable automatic update of frame locals during tracing

Mark Shannon added the comment: While the various frame and debugger PEPs that are open offer a better solution to this, they might not be accepted for 3.11. So I'd like to revisit this. Removing the calls to `PyFrame_FastToLocals` and friends cuts the overhead of tracing down a lot. A

[issue42197] Disable automatic update of frame locals during tracing

Fabio Zadrozny added the comment: > > So, it's expected that `some_module` and `v` would be in the locals at this > > point. > If a function does not have the local variables `some_module` and `v`, then > the change wouldn't be visible to the debugee. So what difference does it make? Right

[issue42197] Disable automatic update of frame locals during tracing

Mark Shannon added the comment: > So, it's expected that `some_module` and `v` would be in the locals at this > point. If a function does not have the local variables `some_module` and `v`, then the change wouldn't be visible to the debugee. So what difference does it make? --

[issue42197] Disable automatic update of frame locals during tracing

Fabio Zadrozny added the comment: @ncoghlan I took a quick look at the PEP... I'm a bit worried about: > On optimised frames, the Python level f_locals API will become a direct > read/write proxy for the frame's local and closure variable storage, and > hence no longer support storing

[issue42197] Disable automatic update of frame locals during tracing

Nick Coghlan added the comment: I've updated PEP 558 with a reference back to this ticket as additional motivation (the update also addresses several of the fragility concerns that Mark raised): https://github.com/python/peps/pull/1787 There's still work to be done to bring the reference

[issue42197] Disable automatic update of frame locals during tracing

Fabio Zadrozny added the comment: I agree that it can be made better, but I think most of the issues right now comes from CPython trying to automatically do something that's not bound to work (calling PyFrame_FastToLocals/PyFrame_LocalsToFast under the hood during the tracing call).

[issue42197] Disable automatic update of frame locals during tracing

Mark Shannon added the comment: I don't think so. Tracing is already somewhat fragile, see https://bugs.python.org/issue30744. Making it more complex is likely to add more bugs. PEP 558 should help, as f_locals becomes a proxy. There are some minor issues with PEP 558 (I'm not convinced that

[issue42197] Disable automatic update of frame locals during tracing

STINNER Victor added the comment: Mark: do you think that it's an acceptable trade-off to ask debuggers and profilers to call explicitly PyFrame_FastToLocalsWithError() and PyFrame_LocalsToFast(), to optimize the common case, when locals are not needed? -- nosy: +Mark.Shannon,

[issue42197] Disable automatic update of frame locals during tracing

STINNER Victor added the comment: Fabio created a thread on python-dev: https://mail.python.org/archives/list/python-...@python.org/thread/62WK3THUDNWZCDOMXXDZFG3O4LIOIP4W/ In 2017, Nick Coghlan wrote the PEP 558 "Defined semantics for locals()". --

[issue42197] Disable automatic update of frame locals during tracing

Change by STINNER Victor : -- nosy: +vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue42197] Disable automatic update of frame locals during tracing

Change by Roundup Robot : -- keywords: +patch nosy: +python-dev nosy_count: 1.0 -> 2.0 pull_requests: +21947 stage: -> patch review pull_request: https://github.com/python/cpython/pull/23028 ___ Python tracker

[issue42197] Disable automatic update of frame locals during tracing

New submission from Fabio Zadrozny : Right now, when a debugger is active, the number of local variables can affect the tracing speed quite a lot. For instance, having tracing setup in a program such as the one below takes 4.64 seconds to run, yet, changing all the variables to have the same