[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: I checked manually cloudpickle_bug.py attached to bpo-43228: it works as expected. The bpo-43228 regression has been fixed. I close again the issue. It was proposed to add a builtins parameter to the types.FunctionType constructor. If someone wants to add

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: New changeset 46496f9d12582bf11f4911ad0f23315d6f277907 by Victor Stinner in branch 'master': bpo-42990: Functions inherit current builtins (GH-24564) https://github.com/python/cpython/commit/46496f9d12582bf11f4911ad0f23315d6f277907 --

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: > Regarding the text in What's New 3.10 about this at > https://docs.python.org/dev/whatsnew/3.10.html#other-language-changes, I > recommend adding there that func.__builtins__ is initialized from > globals["__builtins__"], if it exists, else from the

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Guido van Rossum added the comment: Thanks for the clarifications, Victor! So I now understand: the *identity* of the builtins dict used by a function's code is captured when the function object is created. And I understand why: it's so that in the fast path for the LOAD_GLOBAL opcode we

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: Guido: "I'm still worried about the change in semantics where globals["__builtins__"] is assigned a different dict after the function object has been created (...)" Well, there is a semantics change of Python 3.10 documented at:

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Mark Shannon added the comment: In Python 3.9 the binding is more late-ish binding, than true late binding. Because globals['__builtins__'] is cached for each function activation, executing functions don't see updates. Example: >>> def f(): ... print(len("test")) ... bltns =

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Guido van Rossum added the comment: Thanks, that's clearer. I'm still worried about the change in semantics where globals["__builtins__"] is assigned a different dict after the function object has been created (similar to https://bugs.python.org/file49816/func_builtins2.py). I.e. def

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: Updated PR documentation: --- The types.FunctionType constructor now inherits the current builtins if the globals parameter is used and the globals dictionary has no "__builtins__" key, rather than rather than using {"None": None} as builtins: same behavior

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: I rephrased PR 24564 to clarify the scope of the incompatible change: in practice, only the types.FunctionType constructor changes. Defining functions in Python using "def function(...): ...", eval(code, {}) and exec(code, {}) are not affected. eval() and

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: New changeset 44085a3fc9a150478aec1872dd1079c60dcc42f6 by Victor Stinner in branch 'master': bpo-42990: Refactor _PyFrame_New_NoTrack() (GH-24566) https://github.com/python/cpython/commit/44085a3fc9a150478aec1872dd1079c60dcc42f6 --

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by STINNER Victor : -- pull_requests: +23347 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/24566 ___ Python tracker ___

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: > You say it's on purpose, what's the purpose? Aren't you worried this is going > to break stuff? There is a subtle behavior difference between Python 3.9 and Python 3.10. func_builtins2.py of bpo-43228 works on Python 3.9 but fails on Python 3.10. With my

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Guido van Rossum added the comment: Victor > the new globals has no "__builtins__" key. This change is backward > incompatible on purpose. If someone really wants to run a function in a > different builtins namespace, globals['__builtins__'] must be set explicitly. You say it's on purpose,

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Yury Selivanov added the comment: > So you think that even a dedicated "LEN" opcode would not be any faster? > (This is getting in Paul Sokolovsky territory -- IIRC he has a variant of > Python that doesn't allow overriding builtins.) Yeah, a dedicated LEN opcode could only be faster if it

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by STINNER Victor : -- versions: +Python 3.10 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: I reopen the issue since there is bpo-43228 regression, caused by this issue, which is still under discussion, and Mark also proposed to add a new builtins parameter to the function constructor (FunctionType). I wrote PR 24564 to help fixing bpo-43228

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by STINNER Victor : -- pull_requests: +23345 pull_request: https://github.com/python/cpython/pull/24564 ___ Python tracker ___

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: New changeset a3c3ffa68e6fc4524b1149a6a14d56c3a2e9b612 by Victor Stinner in branch 'master': bpo-42990: Add __builtins__ attribute to functions (GH-24559) https://github.com/python/cpython/commit/a3c3ffa68e6fc4524b1149a6a14d56c3a2e9b612 --

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Guido van Rossum added the comment: > FWIW the globals opcode cache handles all of this now. There's no point in > specifically optimizing the builtins lookup since we optimize all global > lookups for a code object that's hot enough. So you think that even a dedicated "LEN" opcode would

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Yury Selivanov added the comment: > I tried to implement such optimization in my old > https://faster-cpython.readthedocs.io/fat_python.html project. I implemented > guards to de-optimize the code if a builtin is overriden. FWIW the globals opcode cache handles all of this now. There's no

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Guido van Rossum added the comment: Well maybe I'll be picking up those ideas again... Thanks for documenting so much of what you did then! -- ___ Python tracker ___

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

STINNER Victor added the comment: > +1 on exposing f,__builtins__. This change is related to bpo-43228 "Regression in function builtins": cloudpickle is broken by this issue (new PyFunctionObject.func_builtins member). > Of course, the thing I'd really want is a way to state that all

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Guido van Rossum added the comment: +1 on exposing f,__builtins__. Of course, the thing I'd really want is a way to state that all references to builtins are meant to have the exact semantics of those builtins, so the compiler can translate e.g. len(x) into a new opcode that just calls

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by STINNER Victor : -- pull_requests: +23340 pull_request: https://github.com/python/cpython/pull/24559 ___ Python tracker ___

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by Mark Shannon : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed ___ Python tracker ___ ___

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Mark Shannon added the comment: New changeset 0332e569c12d3dc97171546c6dc10e42c27de34b by Mark Shannon in branch 'master': bpo-42990: Further refactoring of PyEval_ functions. (GH-24368) https://github.com/python/cpython/commit/0332e569c12d3dc97171546c6dc10e42c27de34b --

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by Mark Shannon : -- pull_requests: +23192 pull_request: https://github.com/python/cpython/pull/24368 ___ Python tracker ___

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Mark Shannon added the comment: New changeset d6c33fbd346765c6a8654dccacb2338006bf2b47 by Mark Shannon in branch 'master': bpo-42990: Introduce 'frame constructor' struct to simplify API for PyEval_CodeEval and friends (GH-24298)

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by Guido van Rossum : -- nosy: +gvanrossum ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by Mark Shannon : -- keywords: +patch pull_requests: +23121 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/24298 ___ Python tracker

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Mark Shannon added the comment: Rather than: f = create_frame_or_gen(...); if () return f; retval = _PyEval_EvalFrame(tstate, f, 0); _PyObject_GC_TRACK(f); return retval; I was thinking: f = create_frame(...); if () return make_gen(f); retval = _PyEval_EvalFrame(tstate, f,

[issue42990] Improve the C code for calling Python code: _PyEval_EvalCode()

Change by STINNER Victor : -- title: Improve the C code for calling Python code -> Improve the C code for calling Python code: _PyEval_EvalCode() ___ Python tracker ___