[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Nick Coghlan added the comment: New changeset 124227c95f310d2ecd4b567271ab1919fc7000cb by Nick Coghlan in branch 'main': bpo-44800: Document internal frame naming conventions (GH-32281) https://github.com/python/cpython/commit/124227c95f310d2ecd4b567271ab1919fc7000cb -- ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47204] Ensure PyEval_GetGlobals() doesn't set an exception when returning NULL
Nick Coghlan added the comment: Core dev forum thread for the C API question: https://discuss.python.org/t/subtle-c-api-discrepancy-pyeval-getlocals-vs-pyeval-getglobals-with-no-python-frame/14797 -- ___ Python tracker <https://bugs.python.org/issue47204> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47204] Ensure PyEval_GetGlobals() doesn't set an exception when returning NULL
New submission from Nick Coghlan : While working on the first PR for bpo-44800 I provoked an obscure failure in PyImport_ImportModule by making PyEval_GetGlobals set an exception when returning NULL (as PyEval_GetLocals has done since bpo-18408 was implemented for Python 3.4). This ticket covers adding an embedding test case that: * ensures PyEval_GetGlobals() returns NULL without an exception when no Python frame is active * ensures PyEval_GetLocals() returns NULL and sets an exception when no Python frame is active * ensures PyImport_ImportModule still works when no Python frame is active There's an option to slightly change the behaviour of `PyEval_GetLocals()` to NOT set an exception in the "no Python frame" case, and instead only set the error when there is a Python frame, but something goes wrong when attempting to access the fast locals array (such as a memory allocation failure). -- assignee: ncoghlan messages: 416611 nosy: ncoghlan priority: normal severity: normal stage: test needed status: open title: Ensure PyEval_GetGlobals() doesn't set an exception when returning NULL type: enhancement ___ Python tracker <https://bugs.python.org/issue47204> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Nick Coghlan added the comment: Core dev forum thread: https://discuss.python.org/t/proposal-rename-pyinterpreterframe-struct-as-py-framedata/14213 The conclusion from the forum thread and associated PRs was that any of the further renaming proposed didn't provide sufficient additional clarity to justify further code churn in a semi public API. As a result, the final PR just documents the status quo in the internal C API frame header (since the conventions arising from the code-churn-minimising migration to a split data structure is hard to infer just from reading the code): https://github.com/python/cpython/pull/32281 -- ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Change by Nick Coghlan : -- pull_requests: +30342 pull_request: https://github.com/python/cpython/pull/32281 ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47092] [C API] Add PyFrame_GetVar(frame, name) function
Nick Coghlan added the comment: Mark's suggested frame stack API makes sense to me, but being able to get/set specific values through the full frame API seems like it would be useful even if the lower level API existed - if we don't get one of the proxy PEPs in shape to land in 3.11, trace functions written in C could still use this to avoid materialising the locals dict if they only needed to manipulate specific values. Even after a fast locals proxy is defined, there would still be some saving in skipping creating the proxy object when only accessing known keys. We'd need the name-to-index mapping on the code objects to implement this API efficiently, but that isn't a PEP level change in its own right (the proxy PEPs only mention it because they need it) -- ___ Python tracker <https://bugs.python.org/issue47092> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36692] Unexpected stderr output from test_sys_settrace
Nick Coghlan added the comment: I'm not, so I assume this got cleaned up somewhere along the way. -- resolution: -> out of date stage: needs patch -> resolved status: pending -> closed ___ Python tracker <https://bugs.python.org/issue36692> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46964] The global config should not be stored on each interpreter
Nick Coghlan added the comment: I agree the status quo has its benefits. However, it has the distinct downside that because each interpreter has its own storage for copied settings, other code has to assume the settings *might* be different in each interpreter, even if they're currently always the same. If the invariant bits are moved out, then it clearly indicates when code can safely assume that every interpreter will be seeing the same value. -- ___ Python tracker <https://bugs.python.org/issue46964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47135] Allow decimal.localcontext to accept keyword arguments to set context attributes
Nick Coghlan added the comment: Seems reasonable to me, but I think a full implementation would want to throw an error for keyword args that don't already exist as context attributes (otherwise typos would fail silently) -- ___ Python tracker <https://bugs.python.org/issue47135> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46964] The global config should not be stored on each interpreter
Nick Coghlan added the comment: Moving the invariant bits would certainly make sense. As Victor notes, though, some things (like whether to install the signal handlers) should be different. -- ___ Python tracker <https://bugs.python.org/issue46964> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue46761] functools.update_wrapper breaks the signature of functools.partial objects
Nick Coghlan added the comment: On the historical front, wraps & update_wrapper were only designed to handle true wrapper functions (i.e. those that don't change the calling signature). For anything else (including partial), I considered it unlikely that the doc string would still be accurate, let alone any of the other metadata, so I didn't worry about supporting them. That sais, if it's practical to make the results of combining the two less broken then I agree it would make sense to do so. -- ___ Python tracker <https://bugs.python.org/issue46761> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32642] add support for path-like objects in sys.path
Nick Coghlan added the comment: The import system is already complex, so I think the hesitation about allowing arbitrary Path-like objects is warranted. (For example: are importlib's caching semantics really valid for *arbitrary* path-like objects? An object can be path-like without being immutable) Coercion on input (as Noam suggests) would have a much lower risk of unwanted side effects, as any dynamic behaviour would be resolved at insertion time. -- ___ Python tracker <https://bugs.python.org/issue32642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Change by Nick Coghlan : -- pull_requests: +30113 pull_request: https://github.com/python/cpython/pull/32024 ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Change by Nick Coghlan : -- pull_requests: +30078 pull_request: https://github.com/python/cpython/pull/31987 ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue47065] test_curses fails if terminal defaults to bright white text (15)
New submission from Nick Coghlan : test_curses fails for me by default (running on Fedora 35 in KDE's Konsole): ``` [ncoghlan@thechalk cpython]$ echo $TERM xterm-256color [ncoghlan@thechalk cpython]$ ./python -m test -u curses test_curses 0:00:00 load avg: 0.88 Run tests sequentially 0:00:00 load avg: 0.88 [1/1] test_curses test test_curses failed -- Traceback (most recent call last): File "/home/ncoghlan/devel/cpython/Lib/test/test_curses.py", line 48, in wrapped test(self, *args, **kwargs) ^^^ File "/home/ncoghlan/devel/cpython/Lib/test/test_curses.py", line 993, in test_use_default_colors self.assertIn(old, [(curses.COLOR_WHITE, curses.COLOR_BLACK), (-1, -1), (0, 0)]) AssertionError: (15, 0) not found in [(7, 0), (-1, -1), (0, 0)] test_curses failed (1 failure) == Tests result: FAILURE == 1 test failed: test_curses Total duration: 466 ms Tests result: FAILURE ``` The active terminal info indicates that the default text colour is indeed bright white (assuming I'm reading the infocmp output correctly), so it feels like (15, 0) is just missing from the set of permissible "old" colour pairs in the test case: ``` [ncoghlan@thechalk cpython]$ infocmp -L | grep initialize_color initialize_color=\E]4;%p1%d;rgb\072%p2%{255}%*%{1000}%/%2.2X/%p3%{255}%*%{1000}%/%2.2X/%p4%{255}%*%{1000}%/%2.2X\E\, ``` -- components: Tests messages: 415539 nosy: ncoghlan, petr.viktorin, twouters priority: normal severity: normal stage: needs patch status: open title: test_curses fails if terminal defaults to bright white text (15) type: behavior ___ Python tracker <https://bugs.python.org/issue47065> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue21436] Consider leaving importlib.abc.Loader.load_module()
Nick Coghlan added the comment: FWIW, I think it would be desirable to retain/restore some form of API that allows the creation of modules from files without requiring the user to know about module specs (or loaders, or anything else). My current preference would be for a "module_from_file_location" counterpart to "spec_from_file_location" that implements Brett's recipe from above. That code is visually short, but conceptually very dense,and hence far from being obvious to most people trying to make the "path to module" leap. Such a function could also cross-reference runpy.run_path for cases where don't want a module at all, just the top level namespace. -- ___ Python tracker <https://bugs.python.org/issue21436> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45155] Add default arguments for int.to_bytes()
Nick Coghlan added the comment: Never mind, I've forced network byte order in struct strings for so long I had forgotten that native byte order was also the default there. Hence I withdraw that objection. -- ___ Python tracker <https://bugs.python.org/issue45155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45155] Add default arguments for int.to_bytes()
Nick Coghlan added the comment: Rather than defaulting to sys.byteorder, could the byte order default to None and only be optional when not needed? (input value fits in a single byte, output is a single byte) Otherwise the difference in defaults between this method and the struct module (network byte order rather than host byte order) could be very confusing. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue45155> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue39452] Improve the __main__ module documentation
Nick Coghlan added the comment: These changes are excellent - thanks for the patch! Something even the updated version doesn't cover yet is directory and zipfile execution, so I filed bpo-45149 as a follow up ticket for that (the info does exist elsewhere in the documentation, so it's mostly just a matter of adding it to the newly expanded page, and deciding what new cross-references, if any, would be appropriate) -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue39452> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45149] Cover directory and zipfile execution in __main__ module docs
New submission from Nick Coghlan : bpo-39452 covered significant improvements to the __main__ module documentation at https://docs.python.org/dev/library/__main__.html, making it a better complement to the CPython CLI documentation at https://docs.python.org/dev/using/cmdline.html#command-line This follow up ticket covers the `__main__` variant that the previous ticket didn't cover: directory and zipfile execution (which underlies the utility of the `zipapp` stdlib module: https://docs.python.org/dev/library/zipapp.html) The relevant info is present in the CLI and runpy.run_path documentation, so this ticket just covers adding that info to the `__main__` module docs. -- messages: 401440 nosy: ncoghlan priority: normal severity: normal stage: needs patch status: open title: Cover directory and zipfile execution in __main__ module docs type: enhancement ___ Python tracker <https://bugs.python.org/issue45149> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue45020] Freeze all modules imported during startup.
Nick Coghlan added the comment: For the module metadata problem: one potential approach to that for "designed to be frozen" stdlib modules is to set the values directly in the module code, rather than trying to set them automatically in the frozen import machinery. It should also be possible to delete the implicitly created metadata fields and use module level dynamic attribute retrieval to find the stdlib source code for tracebacks and introspection purposes without incurring any start up costs. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue45020> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42538] AsyncIO strange behaviour
Change by Nick Coghlan : -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> asyncio.create_task() documentation should mention user needs to keep reference to the task ___ Python tracker <https://bugs.python.org/issue42538> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42538] AsyncIO strange behaviour
Nick Coghlan added the comment: While filed later, issue44665 correctly describes the problem reported here: asyncio is behaving as expected, but the docs don't make it clear that you need to save a reference to the result of ensure_future()/create_task() if you don't want to risk the task unexpectedly disappearing mid execution. So marking this as a duplicate of that docs issue. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue42538> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Change by Nick Coghlan : -- pull_requests: +26326 pull_request: https://github.com/python/cpython/pull/3640 ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Nick Coghlan added the comment: >From a naming convention perspective, the code comments and NEWS entry in the >PR now refer to "full frame objects" (``PyFrameObject``) and "frame data >storage structs" (``_Py_framedata``) to avoid giving the misleading impression >that introspection and execution frames are different Python types. -- ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename InterpreterFrame to `_Py_framedata`
Nick Coghlan added the comment: PR has been updated with a new API proposal prompted by Mark's review comments on the original proposal. * Rename "pycore_frame.h" to "pycore_framedata.h" * Rename the _interpreter_frame struct to _Py_execution_frame * Rename the type from InterpreterFrame to _Py_framedata * Rather than 6 fields with no prefix, and 6 fields with the "f_" prefix, _Py_framedata fields now consistently have no prefix (globals, builtins, locals, code, lasti, and stack are affected by this) * Use "fdata" rather than a mixture of "frame" and "f" for frame data variables * Generators and coroutines use ``gi_fdata`` (etc) rather than ``gi_xframe`` as their field name * Frame objects use ``f_fdata`` rather than ``f_frame`` as their field name * Thread states use ``fdata`` rather than ``frame`` as their field name * Consistently use _Py_frameobject as the access function prefix, rather than a confusing mixture of _PyFrame and _PyInterpreterFrame * Leave the name of _PyThreadState_PushFrame alone * Leave the name of _PyThreadState_PopFrame alone -- title: Code readability: rename interpreter frames to execution frames -> Code readability: rename InterpreterFrame to `_Py_framedata` ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
Nick Coghlan added the comment: Mark raised some valid concerns with the proposed naming convention over on the PR: * the proposed names make it sound like there are genuinely two kinds of frame, when the actual relationship is between a frame's data storage and a Python object providing an interface to that storage * _PyExecFrame looks like an actual Python type name, so we probably want something more like "Py_buffer" (where the lowercase name is intended to indicate that the struct isn't a full Python type) Mark offered "activation record" as a possible name, but I'm going to see how "_Py_framedata" looks first (with "fdata" as the abbreviated form, since "fd" is so frequently used to mean "file descriptor") I'm also going to see how the PR looks if both the frame and frame data struct keep their existing field prefixes - it may be that changing variable names will be enough to avoid ambiguity, in which case leaving the field names alone genuinely reduces code churn. -- ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43838] There is a way to access an underlying mapping in MappingProxyType
Nick Coghlan added the comment: That resolution makes sense to me as well. Should we make a note in the documentation for https://docs.python.org/3/library/types.html#types.MappingProxyType that the "read-only" protection is to guard against *accidental* modification, not against active attempts to subvert the protection? -- ___ Python tracker <https://bugs.python.org/issue43838> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37398] contextlib.ContextDecorator decorating async functions
Nick Coghlan added the comment: You are correct that this proposal is technically rejected, rather than being a true duplicate of the AsyncContextManager addition. The core problem with the automatic inference idea is that it isn't possible for the standard library to reliably distinguish at decoration time between "synchronous function that happens to return an awaitable" and "coroutine factory function written as a synchronous function that should nevertheless be treated like a coroutine function". That ambiguity doesn't exist when ContextDecorator and AsyncContextDecorator are applied to distinct helper functions. Within a given codebase, you may be able to write a "coroutine argument detection" function that is robust enough for that particular codebase (and iscoroutinefunction() may even be adequate for that purpose), in which case ContextDecorator and AsyncContextDecorator provide the building blocks for implementing the two halves of the solution, leaving only the auto-switching code as project specific. -- resolution: duplicate -> rejected ___ Python tracker <https://bugs.python.org/issue37398> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43838] There is a way to access an underlying mapping in MappingProxyType
Nick Coghlan added the comment: Ouch, you're right - I forgot that dict just returned NotImplemented from comparisons and unions and made it the other operand's problem: >>> d1 = dict(x=1) >>> d2 = dict(x=1) >>> from types import MappingProxyType as proxy >>> d1.__eq__(proxy(d2)) NotImplemented >>> d1.__or__(proxy(d2)) NotImplemented Perhaps we could mitigate the performance hit by skipping the copy when the other side is either a builtin dict (subclasses don't count), or a proxy around a builtin dict? -- ___ Python tracker <https://bugs.python.org/issue43838> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue22240] argparse support for "python -m module" in help
Nick Coghlan added the comment: (Note: this is an old enough ticket that it started out with patch files on the tracker rather than PRs on GitHub. That's just a historical artifact, so feel free to open a PR as described in the devguide if you would prefer to work that way) There are two common ways of solving import bootstrapping problems that affect the build process: 1. Make the problematic import lazy, so it only affects the specific operation that needs the binary dependency; or 2. Wrap a try/except around the import, and do something reasonable in the failing case However, I think in this case it should be possible to avoid the zipfile dependency entirely, and instead make the determination based on the contents of __main__ and __main__.__spec__ using the suggestions I noted in an earlier comment (see https://docs.python.org/3/library/importlib.html#importlib.machinery.ModuleSpec for the info the module spec makes available). This should also solve a problem I believe the current patch has, where I think directory execution will give the wrong result (returning "python -m __main__" as the answer instead of the path to the directory containing the __main__.py file). The three cases to cover come from https://docs.python.org/3/using/cmdline.html#interface-options: * if `__main__.__spec__` is None, the result should continue to be `_os.path.basename(_sys.argv[0])` (as it is in the patch) * if `__main__.__spec__.name` is exactly the string "__main__", `__main__.__spec__.parent` is the empty string, and `__main__.__spec__.has_location` is true, and sys.argv[0] is equal to `__main__.__spec__.origin` with the trailing `__main__.py` removed, then we have a directory or zipfile path execution case, and the result should be a Python path execution command using f'{py} {arg0} * otherwise, we have a `-m` invocation, using f'{py} -m {mod.__spec__.name.removesuffix(".__main__")}' The patch gets the three potential results right, but it gets the check for the second case wrong by looking specifically for zipfiles instead of looking at the contents of __main__.__spec__ and seeing if it refers to a __main__.py file located inside the sys.argv[0] path (which may be a directory, zipfile, or any other valid sys.path entry). For writing robust automated tests, I'd suggest either looking at test.support.script_helper for programmatic creation of executable zipfiles and directories ( https://github.com/python/cpython/blob/208a7e957b812ad3b3733791845447677a704f3e/Lib/test/support/script_helper.py#L209 ) or else factoring the logic out such that there is a helper function that receives "__main__.__spec__" and "sys.argv[0]" as parameters, allowing the test suite to easily control them. The latter approach would require some up front manual testing to ensure the behaviour of the different scenarios was being emulated correctly, but would execute a lot faster than actually running subprocesses would. -- ___ Python tracker <https://bugs.python.org/issue22240> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44590] Create frame objects lazily when needed
Nick Coghlan added the comment: The newly linked pull request isn't actually for this ticket, it's for bpo-44800, a follow-up refactoring proposal related to the variable, struct field, and API naming schemes used for the new lighter weight execution frames. However, the commit message and PR description refer back to this ticket as well, so the RoundUp automation picked it up. -- ___ Python tracker <https://bugs.python.org/issue44590> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
Nick Coghlan added the comment: PR for this proposed refactoring is now up, with a review requested from Mark: https://github.com/python/cpython/pull/27525/ The PR mostly follows what I originally posted, except that I went with _Py_execution_frame and _PyExecFrame for the struct and typedef names, since these are present in a public header. The outdated GDB hooks I found were in the gdbinit example file, rather than the actual libpython.py file that test_gdb covers. I suspect the gdbinit example will need further adjustments to actually work with Python 3.11, so rather than fully updating the implementation dependent pieces, I just added a comment suggesting it be checked after the interpreter optimisation development for 3.11 settles down. -- ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
Change by Nick Coghlan : -- keywords: +patch pull_requests: +26040 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/27525 ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44590] Create frame objects lazily when needed
Change by Nick Coghlan : -- nosy: +ncoghlan nosy_count: 3.0 -> 4.0 pull_requests: +26041 pull_request: https://github.com/python/cpython/pull/27525 ___ Python tracker <https://bugs.python.org/issue44590> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
Nick Coghlan added the comment: As a side effect of working on this, I noticed some changes that are needed to adapt the GDB integration hooks to the new frame state layout. -- nosy: +pablogsal ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
Change by Nick Coghlan : -- assignee: -> ncoghlan ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
Change by Nick Coghlan : -- dependencies: +Create frame objects lazily when needed ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44800] Code readability: rename interpreter frames to execution frames
New submission from Nick Coghlan : When merging the bpo-44590 changes into my PEP 558 implementation branch, I found it very hard to follow when the code was referring to the new interpreter frames rather than the existing Python frame objects that were historically used for both execution and introspection. The "interpreter frame" name was also a little confusing, since the introspection frames are still associated with a specific interpreter, they're just not required for code execution anymore, only for code introspection APIs that call for a full Python object. So, inspired by the "gi_xframe" (etc) attributes added in https://github.com/python/cpython/pull/27077, I'm proposing the following internal refactoring: * Rename "pycore_frame.h" to "pycore_xframe.h" * Rename the _interpreter_frame struct to _execution_frame * Rename the type from InterpreterFrame to ExecFrame * Use "xf_" rather than "f_" as the struct field prefix on execution frames * Use "xframe" and "xf" rather than "frame" and "f" for execution frame variables * Consistently use _PyExecFrame as the access function prefix, rather than a confusing mixture of _PyFrame and _PyInterpreterFrame * Rename _PyThreadState_PushFrame to _PyThreadState_PushExecFrame * Rename _PyThreadState_PopFrame to _PyThreadState_PopExecFrame -- messages: 398675 nosy: Mark.Shannon, ncoghlan priority: normal severity: normal stage: needs patch status: open title: Code readability: rename interpreter frames to execution frames type: enhancement versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44800> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44405] add program passed as string to dis module.
Nick Coghlan added the comment: I suspect the only reason the dis CLI isn't documented is because it's been around for so long (22 years!) that nobody previously noticed it was missing: https://github.com/python/cpython/commit/1fdae12c93246fcf4abbf882ba08df789070dfcc Folks then either didn't know about it, or already knew how to use it. Even the migration to use argparse was a code cleanup from a user that happened to be reading the module source code and noticed that it didn't already do so: https://bugs.python.org/issue18538 I can see value in supporting `-c` and `-m` options to dis to align with the main interpreter CLI. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue44405> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43838] There is a way to access an underlying mapping in MappingProxyType
Nick Coghlan added the comment: Delegating operations to the underlying mapping is still OK, all we're wanting to bypass is the operand coercion dances in abstract.c and object.c that may expose the underlying mapping to the *other* operand. We don't want to implicitly copy though, as the performance hit can be non-trivial for large mappings, even if the algorithmic complexity doesn't change. For the `nb_or` slot, it should be correct to retrieve and call `__or__` from the underlying mapping when the left operand is a MappingProxy instance, and `__ror__` when the right operand is a MappingProxy instance (but the left operand is not). Rich comparison is messier though, since PyObject_RichCompare handles both the operand coercion dance *and* the mapping of the specific comparison operations to their implementation slots. I don't see a clean solution to that other than defining a new PyObject_DelegateRichCompare helper function for the mapping proxy to use that provides the mapping from specific operands to their implementation slots *without* doing the coercion dance. However we resolve it, I think it probably won't be worth the risk of backporting the change (this has been the way it currently is for a long time, and it's a "reduce the risk of error" feature rather than any kind of security boundary). -- ___ Python tracker <https://bugs.python.org/issue43838> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44561] Some expired hyperlinks in Python documentation
Change by Nick Coghlan : -- stage: patch review -> backport needed ___ Python tracker <https://bugs.python.org/issue44561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44561] Some expired hyperlinks in Python documentation
Nick Coghlan added the comment: New changeset b494685b2548489efcc66993cc6c13b027ce3b26 by Steven Hsu in branch 'main': bpo-44561: Update hyperlinks in Doc/distributing/index.rst (#27032) https://github.com/python/cpython/commit/b494685b2548489efcc66993cc6c13b027ce3b26 -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue44561> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43838] There is a way to access an underlying mapping in MappingProxyType
Nick Coghlan added the comment: I stumbled across this independently in bpo-44596, but missed this existing report due to the specific word I was looking for (encapsulation). In addition to the comparison operand coercion, there's now another access option through the `__ror__` method. The bug in both cases arises from delegating a method/function implementation to the underlying mapping type in a way that invokes the full operand coercion dance. (PyObject_RichCompare in one case, PyNumber_Or in the other) Delegating these operations to the underlying mapping does make sense, but it needs to be a lower level delegation that bypasses the operand coercion dance, so the other operand only ever sees the proxy object, not the underlying mapping. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue43838> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44596] Operand coercion breaks MappingProxyType encapsulation
New submission from Nick Coghlan : The fast locals proxy implementation for PEP 558 used the existing MappingProxyType implementation as a starting point. This meant I noticed the following code in the mapping proxy comparison implementation: == static PyObject * mappingproxy_richcompare(mappingproxyobject *v, PyObject *w, int op) { return PyObject_RichCompare(v->mapping, w, op); } == Seeing that code lead to trying the following experiment: == >>> from types import MappingProxyType >>> d = {} >>> proxy = MappingProxyType(d) >>> class Sneaky: ... def __eq__(self, other): ... if other is d: ... raise RuntimeError("Broke encapsulation") ... >>> proxy == Sneaky() Traceback (most recent call last): File "", line 1, in File "", line 4, in __eq__ RuntimeError: Broke encapsulation == The new `__or__` implementation has a corresponding loophole: >>> class SneakyOr: ... def __or__(self, other): ... if other is d: ... raise RuntimeError("Broke encapsulation") ... def __ror__(self, other): ... return self.__or__(other) ... >>> proxy | SneakyOr() Traceback (most recent call last): File "", line 1, in File "", line 6, in __ror__ File "", line 4, in __or__ RuntimeError: Broke encapsulation Delegating these operations to the underlying mapping via the abstract operation layer means that the underlying mapping gets exposed to the operand coercion machinery and can hence be accessed by the other operand. -- messages: 397242 nosy: ncoghlan priority: normal severity: normal stage: needs patch status: open title: Operand coercion breaks MappingProxyType encapsulation type: behavior versions: Python 3.11 ___ Python tracker <https://bugs.python.org/issue44596> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42197] Disable automatic update of frame locals during tracing
Nick Coghlan added the comment: I've updated PEP 558 with a reference back to this ticket as additional motivation (the update also addresses several of the fragility concerns that Mark raised): https://github.com/python/peps/pull/1787 There's still work to be done to bring the reference implementation into line with the updated proposal, but I'm making it an explicit goal to get the proposal submitted for pronouncement in time for inclusion in 3.11. -- versions: +Python 3.11 -Python 3.10 ___ Python tracker <https://bugs.python.org/issue42197> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44515] contextlib test incompatibility with non-refcounted GC
Change by Nick Coghlan : -- keywords: +patch pull_requests: +25486 stage: commit review -> patch review pull_request: https://github.com/python/cpython/pull/26910 ___ Python tracker <https://bugs.python.org/issue44515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44515] contextlib test incompatibility with non-refcounted GC
Change by Nick Coghlan : -- versions: +Python 3.10, Python 3.11 ___ Python tracker <https://bugs.python.org/issue44515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue44515] contextlib test incompatibility with non-refcounted GC
New submission from Nick Coghlan : Backporting the latest contextlib module and test suite to contextlib2, I ran into a couple of CI failures on PyPy3. Investigation showed that a couple of the new test cases were assuming the use of a refcounted GC. One could be fixed by switching to using a synchronous context manager instead of a ``__del__`` method, but the other needed a few explicit gc.collect() calls. -- assignee: ncoghlan keywords: 3.10regression messages: 396545 nosy: ncoghlan, yselivanov priority: normal severity: normal stage: commit review status: open title: contextlib test incompatibility with non-refcounted GC type: behavior ___ Python tracker <https://bugs.python.org/issue44515> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue37398] contextlib.ContextDecorator decorating async functions
Nick Coghlan added the comment: bpo-40816 took the much simpler approach of introducing a separate contextlib.AsyncContextDecorator class. -- resolution: -> duplicate stage: -> resolved status: open -> closed superseder: -> Add missed AsyncContextDecorator to contextlib versions: +Python 3.10 -Python 3.9 ___ Python tracker <https://bugs.python.org/issue37398> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42972] [C API] Heap types (PyType_FromSpec) must fully implement the GC protocol
Nick Coghlan added the comment: New changeset 59af59c2dfa52dcd5605185263f266a49ced934c by Erlend Egeberg Aasland in branch 'main': bpo-42972: Fully support GC for pyexpat, unicodedata, and dbm/gdbm heap types (GH-26376) https://github.com/python/cpython/commit/59af59c2dfa52dcd5605185263f266a49ced934c -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue42972> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: After the next docs build, the documentation for the explicit AST nodes should be available https://docs.python.org/dev/library/ast.html#pattern-matching -- resolution: -> fixed stage: commit review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41682] [Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure
Nick Coghlan added the comment: Another instance at https://github.com/python/cpython/pull/25585/checks?check_run_id=2447701034 I unlinked the previously linked PR, as that looked to be for bpo-43539 rather than this issue. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue41682> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41682] [Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure
Change by Nick Coghlan : -- stage: patch review -> needs patch ___ Python tracker <https://bugs.python.org/issue41682> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue41682] [Windows] test_asyncio: Proactor test_sendfile_close_peer_in_the_middle_of_receiving failure
Change by Nick Coghlan : -- pull_requests: -24157 ___ Python tracker <https://bugs.python.org/issue41682> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43312] Interface to select preferred "user" or "home" sysconfig scheme for an environment
Change by Nick Coghlan : -- resolution: -> fixed stage: patch review -> resolved status: open -> closed type: -> enhancement ___ Python tracker <https://bugs.python.org/issue43312> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Change by Nick Coghlan : -- assignee: ncoghlan -> brandtbucher ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Change by Nick Coghlan : -- stage: patch review -> commit review ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: PR is green now, and I don't have any further changes I want to make, so Brandt, please feel free to merge if you're happy with the proposed AST nodes, and the way I've gone about implementing them. If you'd like some adjustments, it probably makes the most sense to just amend the PR directly - otherwise the 24-hour turnaround on comments due to the time zone difference will potentially give us trouble with the beta deadline. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: PR against the main repo is now up: https://github.com/python/cpython/pull/25585 A couple of things in the unparser tripped me up (ast_unparse.c only handles annotation expressions, the way ast._Unparser.visit() is defined means NodeVisitor.generic_visit can't be used), so the PR also includes some clarifying comments for future editors. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Change by Nick Coghlan : -- pull_requests: +24305 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/25585 ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: Working on the unparser picked up an inconsistency I had previously missed: aside from named expressions, "target" attributes in the AST are generally full expression subnodes, rather than identifier attributes. Identifier attributes on nodes are typically called "name", including in the originally implemented MatchAs node. Accordingly, I've switched both MatchStar and MatchAs over to having "name" attributes instead of "target" attributes. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: Interesting idea - I had missed that you were suggested "identifier*" for the cls node. It's simple enough to check for Name-or-Attribute in the AST validator though, and sticking with a standard expr_ty node means getting a lot of standard handling in the rest of the compiler and the unparser. My commitments today finished earlier than I expected, so I'm about to tackle the unparser now. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: Setting clear timing expectations, given beta is so close: I'm not expecting to have time to finish the unparser work over the weekend, but I do expect to have time to finish it on Monday evening my time (UTC+10). -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: https://github.com/ncoghlan/cpython/pull/9/commits/54940a3817df3046da3f9c51d4d426a73b2ec786 implements Brandt's suggestion of disallowing NULL keys in MatchMapping and instead passing in an explicit "rest" identifier to capture the extra keys. This did require moving to a multi-branch pattern definition in the grammar, similar to class patterns, in order to enforce the separating comma when both ordinary key:pattern pairs and a double-star target are present. I think it's a nice ergonomic improvement on the AST node itself though, as it makes it trivial to check if a mapping patterns captures extra keys or not (vs the relatively non-obvious check to see if the last key is NULL/None). That just leaves updating the unparser to handle the new node types. Capturing the latest node definitions from the draft PR: pattern = MatchAlways | MatchValue(expr value) | MatchSingleton(constant value) | MatchSequence(pattern* patterns) | MatchMapping(expr* keys, pattern* patterns, identifier? rest) | MatchClass(expr cls, pattern* patterns, identifier* kwd_attrs, pattern* kwd_patterns) | MatchStar(identifier? target) -- The optional "rest" MatchMapping parameter handles capturing extra mapping keys | MatchAs(pattern? pattern, identifier target) | MatchOr(pattern* patterns) attributes (int lineno, int col_offset, int end_lineno, int end_col_offset) -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: To get the "0+0" matching case rejected again I had to flesh out a bit more of the AST validator. This initial piece of the validator replaces the previously pattern-matching-specific AST optimisation code that refused to fold BinOp expressions that weren't complex numbers, allowing the compiler to reject them. I also added two new tests (240b and 241b) to ensure that 0+0 & f"" were also rejected as mapping keys. 241b passes with the current PR, as f"" gets rejected by both the check in the AST validator *and* by the "constant or attribute lookup" restriction in the compiler, so the latter check covers for the leaky validator. 240b is disabled for now, as it won't pass until the AST validator checks all the subexpressions being used as keys in a mapping pattern (because it gets constant folded to "0", the check in the compiler accepts it). That leaves fixing the unparser tests as the key thing to do before I create the main PR. Before that, though, I'll look at potentially simplifying the MatchMapping code by changing the signature as Brandt suggested. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: Segfaults are fixed (I had a couple of cases where I wasn't checking for NULL, and another where I was looking up MatchMapping attributes on a MatchClass node). test_patma passes except for test cases 240, 241, and 242, which look like genuine regressions - the logic to reject illegal MatchValue nodes is missing from the code generator side, and these particular values make it through the parser OK. I'll need to add back some of the code I took out so these still get rejected. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: My proposed MatchConstant node name was bothering me, as most constants are actually matched by equality in MatchValue, the same way variables are. So I'm switching my proposed name for that node to be MatchSingleton, since that better reflects the key difference with MatchValue (i.e. comparing by identity rather than by value, and specifically comparing with the builtin singletons). -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: I've removed the "?" from the end attributes for pattern nodes. The draft PR branch now compiles, but I've clearly made a mistake somewhere as it segfaults when trying to compile test_patma.py. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43897] Implement support for validation of pattern matching ASTs
Change by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue43897> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43897] Implement support for validation of pattern matching ASTs
Change by Nick Coghlan : -- priority: normal -> critical ___ Python tracker <https://bugs.python.org/issue43897> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: The draft PR moved because I messed up the previous branch name: https://github.com/ncoghlan/cpython/pull/9/files It's to the point where everything except symtable.c compiles. I put an initial skeleton of a validator in, but only enough to check that the parallel sequence lengths in the class and mapping pattern nodes are consistent - there's still plenty to be done in Batuhan's PR (e.g. because the AST constant folding now just delegates to the expression folding functions for MatchValue values and MatchMapping keys, this iteration only enforces the restrictions on permitted subexpressions in the surface syntax). In addition to Brandt's MatchStar suggestion, I've also tweaked the MatchClassparameter names to better match PEP 634 (the old names were based on PEP 642's attribute pattern concept, so "extra_*" made sense, but "kwd_*" makes more sense for PEP 634): pattern = MatchAlways | MatchValue(expr value) | MatchConstant(constant value) | MatchSequence(pattern* patterns) | MatchMapping(expr* keys, pattern* patterns) | MatchClass(expr cls, pattern* patterns, identifier* kwd_attrs, pattern* kwd_patterns) | MatchStar(identifier? target) -- A NULL entry in the MatchMapping key list handles capturing extra mapping keys | MatchAs(pattern? pattern, identifier target) | MatchOr(pattern* patterns) attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) I think the idea of making the MatchMapping node signature "MatchMapping(expr* keys, pattern* patterns, identifier? rest)" has merit, but I'd like to get this initial iteration reviewed first to minimise the diff in the compiler_pattern_mapping implementation (right now it is fairly clear that the code generation for mapping patterns hasn't changed, but that will become significantly less obvious if/when the "**rest" handling changes away from working the same way _PyAST_Dict works) -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Change by Nick Coghlan : -- stage: patch review -> needs patch ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Change by Nick Coghlan : -- keywords: +patch pull_requests: +24241 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/25521 ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: https://www.python.org/dev/peps/pep-0642/#representing-patterns-explicitly-in-the-abstract-syntax-tree covers the rationale for why it is potentially problematic to reuse expression nodes for match patterns: it doesn't semantically align with the fact that patterns are *not* expressions, so the AST ends up being closer to a concrete syntax tree than we would like. For the specifics of the AST nodes: * restricting sub expressions directly in the AST itself is tricky without side effects on the AST for non-pattern-matching code, so I'm intending to add any such restrictions that we want to enforce to the validator (other statements and expressions like yield, await, return, break, and continue already have separately enforced restrictions like that) * cls needs to be an expression node to allow for attribute lookups * the linked draft PR already updates the grammar to emit the new AST. MatchClass is able to collect the keyword args and their corresponding patterns in roughly the same way that MatchMapping collects its key:pattern pairs -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: It's specifically the definition of "match_case" in the AST that is affected: match_case = (pattern pattern, expr? guard, stmt* body) pattern = MatchAlways | MatchValue(expr value) | MatchConstant(constant value) | MatchSequence(pattern* patterns) | MatchMapping(expr* keys, pattern* patterns) | MatchClass(expr cls, pattern* patterns, identifier* extra_attrs, pattern* extra_patterns) | MatchRestOfSequence(identifier? target) -- A NULL entry in the MatchMapping key list handles capturing extra mapping keys | MatchAs(pattern? pattern, identifier target) | MatchOr(pattern* patterns) attributes (int lineno, int col_offset, int? end_lineno, int? end_col_offset) Relative to the PEP 642 AST, the notion of a "matchop" is gone - MatchValue always compares by equality, and there's a MatchConstant node for the identity comparisons against None, True, and False. The grammar, code generator, AST validator, and unparser are then updated to produce or consume the new AST nodes. -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
Nick Coghlan added the comment: Agreed. I had wanted the AST to be part of the PEPs specifically *because* it's a public API, but didn't check until today whether or not the original PEP 634 implementation had been merged as-is, or with a cleaned up AST definition. https://github.com/ncoghlan/cpython/pull/8/files is the initial implementation with the AST and Grammar file updated. I'll only create the CPython PR once the tests are passing, but I'll try to give Brandt at least a week to review it (I'll also ping him directly via email). -- ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43892] Make match patterns explicit in the AST
New submission from Nick Coghlan : In the SC submission ticket for PEP 642 (https://github.com/python/steering-council/issues/43 ), Guido indicated he was in favour of the more explicit pattern matching AST changes suggested in that PEP. This ticket covers adapting the pattern matching implementation to explicitly separate pattern nodes from expr nodes in the AST, so the code generator doesn't need to be context dependent. -- assignee: ncoghlan messages: 391430 nosy: brandtbucher, ncoghlan priority: normal severity: normal stage: needs patch status: open title: Make match patterns explicit in the AST type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue43892> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue43133] Accept file mode "rw" to open-or-create a file and seek to the start
New submission from Nick Coghlan : After yet again trying to use "rw" to open a file in read/write mode, having it fail, and then having to look up the docs to remind myself that the correct spelling is "r+", I'm finally filing this to suggest we make the obvious "rw" spelling actually work. Rather than making it an alias for any of the existing modes, I'd suggest that it combine the features of "r+" and "a+": * like "r+" it would start the file pointer at the beginning of the file * like "a+" it would create the file if it didn't exist, while leaving existing files alone -- messages: 386510 nosy: ncoghlan priority: low severity: normal status: open title: Accept file mode "rw" to open-or-create a file and seek to the start type: enhancement ___ Python tracker <https://bugs.python.org/issue43133> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42783] asyncio.sleep(0) idiom is not documented
Nick Coghlan added the comment: I merged the update to the 3.10 docs, but the automated backport to 3.9 to update the online docs failed. -- resolution: -> fixed stage: patch review -> backport needed versions: +Python 3.10, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42783> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42783] asyncio.sleep(0) idiom is not documented
Nick Coghlan added the comment: New changeset 5c30145afb6053998e3518befff638d207047f00 by Simon Willison in branch 'master': bpo-42783: Documentation for asyncio.sleep(0) (#24002) https://github.com/python/cpython/commit/5c30145afb6053998e3518befff638d207047f00 -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue42783> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42455] Add decorator_factory function to functools module
Nick Coghlan added the comment: Sorry, I meant to include this comment as well: The other filter I applied here was "Could someone figure out the boilerplate code themselves in less time than it would take them to find, read, and understand the documentation for the new helper function?" I'm not sure the answer to that question is "Yes" when writing a decorator factory for the first time, but I am confident it is "Yes" when reading a decorator factory, and for anyone writing their second and subsequent factory (especially if they use the "def factory(decorated=None, /, *, keyword="only", params=True)" idiom the way dataclass and _tp_cache do). -- ___ Python tracker <https://bugs.python.org/issue42455> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42455] Add decorator_factory function to functools module
Nick Coghlan added the comment: I think the idea is a plausible and well-presented suggestion, but I'm afraid I'm still going to agree with the view that we shouldn't add this. >From a maintainability point of view, *generically* detecting the difference >between "applied without parentheses as a decorator" and "called with >parameters to produce a partial function to be used as a decorator" isn't >quite as simple as just calling "callable(args[0])", since it's entirely >possible for decorator factories to accept callables as parameters. We could >try to document our way around that problem, but we wouldn't be able to >eliminate it entirely. Given the relative rarity of the use case, and the potential for subtle issues when the assumptions of the decorator_factory decorator aren't met, I'm happy to leave this task to inline mostly-but-not-completely-boilerplate code rather than trying to abstract it away. -- resolution: -> rejected stage: patch review -> resolved status: open -> closed ___ Python tracker <https://bugs.python.org/issue42455> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue33944] Deprecate and remove code execution in pth files
Nick Coghlan added the comment: PEP 648 has been posted with a proposal to migrate sitecustomize.py, usersitecustomize.py and arbitrary code execution in pth files to a directory based `__sitecustomize__` structure: https://www.python.org/dev/peps/pep-0648/ -- ___ Python tracker <https://bugs.python.org/issue33944> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42642] logging: add high priority log level for warnings being cleared
New submission from Nick Coghlan : When using the logging module for long running services, there's one limitation of the predefined logging levels that I semi-regularly run into: the only entirely reliable log level for reporting that a WARNING state has been cleared is itself WARNING. Using INFO instead means that it is common to get error logs that show warning states being entered without any matching "error cleared" notification. My idea for resolving this would be to define a new ESSENTIAL log level between WARNING and ERROR. The idea of the new log level would be to have a logged-by-default level for non-fault-but-essential messages like: * service startup (including version info) * service shutdown * warnings being cleared (NOTICE would be another possible name for such a level) -- messages: 383027 nosy: ncoghlan, vinay.sajip priority: low severity: normal stage: needs patch status: open title: logging: add high priority log level for warnings being cleared type: enhancement versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42642> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42260] [C API] Add PyInterpreterState_SetConfig(): reconfigure an interpreter
Change by Nick Coghlan : -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue42260> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42282] Constant folding is skipped in named expressions
Change by Nick Coghlan : -- resolution: -> fixed status: open -> closed ___ Python tracker <https://bugs.python.org/issue42282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42282] Constant folding is skipped in named expressions
Nick Coghlan added the comment: Since this was only a performance issue, I'm not planning to backport it to earlier releases. -- stage: patch review -> resolved ___ Python tracker <https://bugs.python.org/issue42282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42282] Constant folding is skipped in named expressions
Nick Coghlan added the comment: New changeset 8805a4dad201473599416b2c265802b8885f69b8 by Nick Coghlan in branch 'master': bpo-42282: Fold constants inside named expressions (GH-23190) https://github.com/python/cpython/commit/8805a4dad201473599416b2c265802b8885f69b8 -- ___ Python tracker <https://bugs.python.org/issue42282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42282] Constant folding is skipped in named expressions
Change by Nick Coghlan : -- keywords: +patch pull_requests: +22093 stage: needs patch -> patch review pull_request: https://github.com/python/cpython/pull/23190 ___ Python tracker <https://bugs.python.org/issue42282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42282] Constant folding is skipped in named expressions
Change by Nick Coghlan : -- assignee: -> ncoghlan ___ Python tracker <https://bugs.python.org/issue42282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42282] Constant folding is skipped in named expressions
New submission from Nick Coghlan : While working on the PEP 642 reference implementation I removed the "default:" case from the switch statement in astfold_expr as part of making sure the new SkippedBinding node was being handled everywhere it needed to be. This change picked up that NamedExpr_kind wasn't being handled either, and a check with the dis module confirmed that using the walrus operator turns off constant folding: ``` [ncoghlan@thechalk cpython]$ ./python Python 3.10.0a2+ (heads/pep-642-constraint-patterns-dirty:4db2fbd609, Nov 7 2020, 16:42:19) [GCC 10.2.1 20201016 (Red Hat 10.2.1-6)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import dis >>> dis.dis("1 + 1") 1 0 LOAD_CONST 0 (2) 2 RETURN_VALUE >>> dis.dis("(x := 1 + 1)") 1 0 LOAD_CONST 0 (1) 2 LOAD_CONST 0 (1) 4 BINARY_ADD 6 DUP_TOP 8 STORE_NAME 0 (x) 10 RETURN_VALUE >>> ``` The missing switch statement entry is just: ``` case NamedExpr_kind: CALL(astfold_expr, expr_ty, node_->v.NamedExpr.value); break; ``` Which gives the expected result: ``` [ncoghlan@thechalk cpython]$ ./python -c "import dis; dis.dis('(x := 1 +1)')" 1 0 LOAD_CONST 0 (2) 2 DUP_TOP 4 STORE_NAME 0 (x) 6 RETURN_VALUE ``` -- messages: 380494 nosy: ncoghlan priority: normal severity: normal stage: needs patch status: open title: Constant folding is skipped in named expressions type: performance versions: Python 3.10 ___ Python tracker <https://bugs.python.org/issue42282> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue40952] GCC overflow warnings (format-overflow, stringop-overflow)
Nick Coghlan added the comment: I *think* the lnotab one is the compiler failing to detect that the pointer has been updated to point inside the body of a Python object, but I'm also not 100% sure that it's a false alarm. -- nosy: +ncoghlan ___ Python tracker <https://bugs.python.org/issue40952> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue42186] unittest overrides more serious warnings filter added before unittest.main()
Nick Coghlan added the comment: Closing the old one as partially fixed, and linking here as a superseder makes sense to me, so I went ahead and did that. -- ___ Python tracker <https://bugs.python.org/issue42186> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15626] unittest.main negates -bb option and programmatic warning configuration
Nick Coghlan added the comment: Issue #42186 now covers the request to have a way to tell unittest to leave the warnings filter alone even if it's set programmatically rather than by modifying ``sys.warnoptions``. (Changing version to 3.7 to indicate when the interaction with `-bb` was fixed, rather than the version it was reported against) -- resolution: -> fixed stage: -> resolved status: open -> closed versions: +Python 3.7 -Python 3.6 ___ Python tracker <https://bugs.python.org/issue15626> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15626] unittest.main negates -bb option and programmatic warning configuration
Change by Nick Coghlan : -- superseder: -> unittest overrides more serious warnings filter added before unittest.main() ___ Python tracker <https://bugs.python.org/issue15626> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue32002] test_c_locale_coercion fails when the default LC_CTYPE != "C"
Nick Coghlan added the comment: Removing issue assignment, as I'm no longer actively investigating this. -- assignee: ncoghlan -> ___ Python tracker <https://bugs.python.org/issue32002> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue30672] PEP 538: Unexpected locale behaviour on *BSD (including Mac OS X)
Nick Coghlan added the comment: Removing issue assignment, as I'm not actively investigating this. -- assignee: ncoghlan -> ___ Python tracker <https://bugs.python.org/issue30672> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34206] Move and clarify Py_Main documentation
Nick Coghlan added the comment: Added 3.8 back in to the target versions. However, if the automatic backport doesn't work for that branch, I'll probably skip it rather than fixing any conflicts. -- versions: +Python 3.8 ___ Python tracker <https://bugs.python.org/issue34206> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue34206] Move and clarify Py_Main documentation
Nick Coghlan added the comment: Adjusted target versions, as I never previously got around to merging this PR. -- versions: +Python 3.10, Python 3.9 -Python 3.7, Python 3.8 ___ Python tracker <https://bugs.python.org/issue34206> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com