[Python-Dev] Re: Importing a submodule doesn't always set an attribute on its parent
On 4/8/2022 7:56 PM, Brett Cannon wrote: On Fri, Apr 8, 2022 at 4:38 PM dfremont--- via Python-Dev mailto:python-dev@python.org>> wrote: If you import A.B, then remove A from sys.modules and import A.B again, the newly-loaded version of A will not contain an attribute referring to B. ... for example PLR 5.7 states that "'import XXX.YYY.ZZZ' should expose 'XXX.YYY.ZZZ' as a usable expression". And it did. But then you went behind the curtain and moved stuff around. Finally, it violates the "invariant" stated in PLR 5.4.2 that if 'A' and 'A.B' both appear in sys.modules, then A.B must be defined and refer to sys.modules['A.B']. That isn't an invariant that holds when you delete things outside of the import system; that statement is what the import system /does/, not what the import system guarantees to always be true. ... Feel free to propose some language to update the docs, Perhaps something intentionally vague like "Manual deletion of entries from sys.modules may invalidate statements above, even after re-imports." or "Manual deletion of entries from sys.modules may result in surprising behavior, even after re-imports." -- Terry Jan Reedy ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7CMQMWJJMM7RUDWUQXL3MW64KL4VW3P6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Importing a submodule doesn't always set an attribute on its parent
On Fri, Apr 8, 2022 at 4:38 PM dfremont--- via Python-Dev < python-dev@python.org> wrote: > Hello, > > I came across what seems like either a bug in the import system or a gap > in its documentation, so I'd like to run it by folks here to see if I > should submit a bug report. If there's somewhere else more appropriate to > discuss this, please let me know. > > If you import A.B, then remove A from sys.modules and import A.B again, > the newly-loaded version of A will not contain an attribute referring to B. > Using "collections.abc" as an example submodule from the standard library: > > >>> import sys > >>> import collections.abc > >>> del sys.modules['collections'] > >>> import collections.abc > >>> collections.abc > Traceback (most recent call last): > File "", line 1, in > AttributeError: module 'collections' has no attribute 'abc' > > This behavior seems quite counter-intuitive to me: why should the fact > that B is already loaded prevent adding a reference to it to A? Because `"collections.abc" in sys.modules` is true. The import system expects that if you already imported a module then everything that needed to happen, happened. Basically you cheated by not doing a thorough cleaning of sys.modules by not deleting all the submodules as well. > It also goes against the general principle that "import FOO" makes the > expression "FOO" well-defined; You're dealing with the import system; you never got to have a well-defined statement to begin with. > for example PLR 5.7 states that "'import XXX.YYY.ZZZ' should expose > 'XXX.YYY.ZZZ' as a usable expression". And it did. But then you went behind the curtain and moved stuff around. > Finally, it violates the "invariant" stated in PLR 5.4.2 that if 'A' and > 'A.B' both appear in sys.modules, then A.B must be defined and refer to > sys.modules['A.B']. > That isn't an invariant that holds when you delete things outside of the import system; that statement is what the import system *does*, not what the import system guarantees to always be true. > > On the other hand, PLR 5.4.2 also states that "when a submodule is loaded > using any mechanism... a binding is placed in the parent module's namespace > to the submodule object", which is consistent with the behavior above, > since the second import of A.B does not actually "load" B (only retrieve it > from the sys.modules cache). So perhaps Python is working as intended here, > and there is an unwritten assumption that if you unload a module from the > cache, you must also unload all of its submodules. If so, I think this > needs to be added to the documentation (which currently places no > restrictions on how you can modify sys.modules, as far as I can tell). > > This may be an obscure corner case that is unlikely to come up in practice > (I imagine few people need to modify sys.modules), but it did actually > cause a bug in a project I work on, where it is necessary to uncache > certain modules so that they can be reloaded. I was able to fix the bug > some other way, but I think it would still be worthwhile to either make the > import behavior more consistent (so that 'import A.B' always sets the B > attribute of A) or add a warning in the documentation about this case. I'd > appreciate any thoughts on this! > Feel free to propose some language to update the docs, but changing this behaviour very well may have unintended consequences, so I would rather not try to change it. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/U5DDT4DUJ7U3VO62VZ333SWIN7QFZPHJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Importing a submodule doesn't always set an attribute on its parent
Hello, I came across what seems like either a bug in the import system or a gap in its documentation, so I'd like to run it by folks here to see if I should submit a bug report. If there's somewhere else more appropriate to discuss this, please let me know. If you import A.B, then remove A from sys.modules and import A.B again, the newly-loaded version of A will not contain an attribute referring to B. Using "collections.abc" as an example submodule from the standard library: >>> import sys >>> import collections.abc >>> del sys.modules['collections'] >>> import collections.abc >>> collections.abc Traceback (most recent call last): File "", line 1, in AttributeError: module 'collections' has no attribute 'abc' This behavior seems quite counter-intuitive to me: why should the fact that B is already loaded prevent adding a reference to it to A? It also goes against the general principle that "import FOO" makes the expression "FOO" well-defined; for example PLR 5.7 states that "'import XXX.YYY.ZZZ' should expose 'XXX.YYY.ZZZ' as a usable expression". Finally, it violates the "invariant" stated in PLR 5.4.2 that if 'A' and 'A.B' both appear in sys.modules, then A.B must be defined and refer to sys.modules['A.B']. On the other hand, PLR 5.4.2 also states that "when a submodule is loaded using any mechanism... a binding is placed in the parent module's namespace to the submodule object", which is consistent with the behavior above, since the second import of A.B does not actually "load" B (only retrieve it from the sys.modules cache). So perhaps Python is working as intended here, and there is an unwritten assumption that if you unload a module from the cache, you must also unload all of its submodules. If so, I think this needs to be added to the documentation (which currently places no restrictions on how you can modify sys.modules, as far as I can tell). This may be an obscure corner case that is unlikely to come up in practice (I imagine few people need to modify sys.modules), but it did actually cause a bug in a project I work on, where it is necessary to uncache certain modules so that they can be reloaded. I was able to fix the bug some other way, but I think it would still be worthwhile to either make the import behavior more consistent (so that 'import A.B' always sets the B attribute of A) or add a warning in the documentation about this case. I'd appreciate any thoughts on this! Thanks, Daniel ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VIPXZRK3OJNSVNSZSAJ7CO6QFC2RX27W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
> > Discourse it just flat-out easier to admin: individuals can flag posts, > automatic spam detection, site-wide admins instead of per-list, ability to > split topics, ability to lock topics, ability to "slow down" topics, > time-limited suspensions, etc. I quit being an admin for any ML beyond > python-committers because I found it too frustrating to deal w/ when > compared to the tools I have on discuss.python.org. > Personally, I never found administering mailing lists to be all that challenging. Also, I think it's worth taking into consideration what works best for users, not just admins. There are far more interactions with discussion media by users than administrators. My personal preference as a user is for mailing lists (everything is funneled through the same user interface rather than several not-quite-identical forum and social media interfaces - I do more than just Python stuff online, and suspect many other people do). Still, I understand that I am a dinosaur and the world is changing, so I shouldn't be surprised that a meteor is approaching. Skip > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/X777BTUZ7VFYIJDNP4AIHOR4JYZOJZ4Q/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On Thu, Apr 7, 2022 at 7:33 PM Stephen J. Turnbull < stephenjturnb...@gmail.com> wrote: > Gregory P. Smith writes: > > > We feel it too. We've been finding Discourse more useful from a > community > > moderation and thread management point of view as well as offering > markdown > > text and code rendering. Ideal for PEP discussions. > > The specific mention of "community moderation" and "thread management" > makes me suspect that part of that effect is due to increased cost of > participation for casual observers. As one of the discuss.python.org admins and a former ML admin (sans python-committers), I can tell that "increased cost of participation for casual observers" is not at all the motivation behind Greg's comment (you can look at the user of the Users category to see that the barrier of entry isn't there). Discourse it just flat-out easier to admin: individuals can flag posts, automatic spam detection, site-wide admins instead of per-list, ability to split topics, ability to lock topics, ability to "slow down" topics, time-limited suspensions, etc. I quit being an admin for any ML beyond python-committers because I found it too frustrating to deal w/ when compared to the tools I have on discuss.python.org. Suggestion for PEP monitoring: > > AIUI the PEP process at a high level view is fairly well monitored by > a certain set of GitHub commits: the proto-PEP "PEP-" commit, the > commit that assigns it a PEP number, and commits that change status. > How about a GitHub bot that does nothing but post PEP commit logs to a > dedicated Discourse channel? It should be possible to remove typo > fixes and the like by posting any that change title, number, or > status, and from the rest exclude any commits that change less than a > dozen lines or something like that. Or perhaps PEP committers could > be asked to include some kind of tag like "#trivial" to distinguish > them. > It's a possibility if someone wants to put the work in. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LS5XDBPIRHGBQ6Q2255366BBPQ2HYSEV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
Start up overhead due to imports is a real problem for some class of
applications, e.g. CLIs, and I’ve seen a lot of hacks implemented to get Python
CLIs to be more responsive. E.g. getting from invocation to —help output is a
major UX problem.
It’s often more complicated than just imports alone though. Expensive module
scope initializations and decorators contribute to this problem. Python start
up time is one of the main drivers for rewriting CLIs in Go and other languages
where I work. Note that this is much less of a problem for things like web
services or other long running applications because that start up time is
either amortized over the lifetime of the application, or aren’t directly
visible to the end user.
Lazy imports *might* help with this and seems aligned with the common trick of
moving imports into functions rather than at module scope. Faster CPython
might help too. But these all feel like they aren’t tackling the start up
problem head on[1]. Lots of ideas have been discussed over the years (I
remember some in-depth ones at the Microsoft core sprint a few years ago), and
I’m sure there are all kinds of other tricks that people use.
However, if start up time isn’t a direct benefit of on-demand imports (a.k.a.
declarative imports), I’m not sure how actually useful or used they will be. I
dunno, top-of-module imports never really bothered me that much.
-Barry
[1] I could be wrong about Faster CPython; ISTR there are some tickets on that
project’s tracker that talk about start up times.
> On Apr 8, 2022, at 09:40, Guido van Rossum wrote:
>
> The interesting idea here seems to make "lazy imports" easier to implement by
> making them explicit in the code. So far, most lazy import frameworks for
> Python have done hacks with `__getattribute__` overrides. IIRC the Cinder
> version even modifies the bytecode and/or the interpreter. Disregarding the
> specific notation proposed, *if* people would be willing to mark the points
> where they expect lazy imports explicitly, that would make implementation
> much simpler.
>
> The argument that "imports on top" makes code more readable seems pretty weak
> to me. The current hacks to speed up startup already violate this rule
> (imports inside functions), and in most cases I start reading or writing code
> in the middle of a file (having gotten there via a search in my editor) and
> the meaning of an import is either obvious (e.g. re.match(...)) or requires
> another annoying search to find the definition of a certain unknown variable.
> Tools can easily show all imports a module does.
>
> The key questions to me are
> - What should the notation be?
> - Will users be willing to use it?
>
> --Guido
>
>
>
> On Fri, Apr 8, 2022 at 1:26 AM Malthe wrote:
> This is an idea which has been brought up before, sometimes introduced
> as "heresy". But an interesting twist has surfaced now which is
> typing.
>
> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports, usable anywhere using a simple
> syntax, @.
>
> For example, `some_regex = @re.compile(...)`.
>
> What happens then is that before anything else in that module, that
> symbol is imported:
>
> from re import compile as _mangled_re_compile
>
> It must be the very first thing (hoisting) because when else would it
> happen? It's been suggested before to have a shorthand syntax which
> does a dynamic import at the time of using it but this brings me to
> the twist:
>
> We want typing to pick up these imports. And this twist has a second
> leg which is that we often need to import symbols simply in order to
> type some argument type or return type. This leads to a great many
> more imports to type.
>
> (Nevermind that if you want to take typing further, abstract
> interfaces really make more sense rather than specific
> implementations, but the point is the same.)
>
> A situation where this would come in really handy is in scripting such
> as how we use Python in Apache Airflow to let users write out simple
> workflows. A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
>
> default_args = {
> "start_date": @datetime.datetime(...)
> }
>
> It's a lot more ergonomic from a user perspective (well perhaps for
> some users and for some programs).
>
> Thoughts?
>
> Cheers
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/UX6EJHLJNNLMFPWVPF5ANYHQSHDZK7SV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> ___
> Python-Dev mailing
[Python-Dev] Re: Declarative imports
An interesting point in the lazy imports design space that I hadn't previously considered could be: - lazy imports are explicitly marked and usage of the imported name within the module is transparent, but - lazily imported names are _not_ visible in the module namespace; they can't be accessed by other modules or re-exported; they are internal-use-only within the module This compromise would, I think, make it possible to implement lazy imports entirely in the compiler (effectively as syntax sugar for an inline import at every usage site), which is definitely an implementation improvement. I think in practice explicitly marking lazy imports would make it somewhat harder to gain the benefits of lazy imports for e.g. speeding up startup time in a large CLI, compared to an implicit/automatic approach. But still could be usable to get significant benefits. Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZT6CXQPFCWZD2M65YXCSAPPGNDGA6WNE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2022-04-01 - 2022-04-08)
Python tracker at https://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open7146 ( -7)
closed 51841 (+78)
total 58987 (+71)
Open issues with patches: 2890
Issues opened (49)
==
#47104: Rewrite asyncio.to_thread tests to use IsolatedAsyncioTestCase
https://bugs.python.org/issue47104 reopened by kj
#47136: The variable __module__ in the class body getting an undesirab
https://bugs.python.org/issue47136 reopened by ethan.furman
#47191: inspect - getasyncgeneratorstate, getasyncgeneratorlocals
https://bugs.python.org/issue47191 opened by animatea.programming
#47192: sys._getframe audit event has frame as argument in 3.8-3.10
https://bugs.python.org/issue47192 opened by Dutcho
#47193: Use zlib-ng rather than zlib in binary releases
https://bugs.python.org/issue47193 opened by gregory.p.smith
#47194: Upgrade to zlib v1.2.12 in CPython binary releases
https://bugs.python.org/issue47194 opened by gregory.p.smith
#47195: importlib lock race issue in deadlock handling code
https://bugs.python.org/issue47195 opened by rpurdie
#47197: ctypes mishandles `void` return type
https://bugs.python.org/issue47197 opened by hoodchatham
#47199: multiprocessing: micro-optimize Connection.send_bytes() method
https://bugs.python.org/issue47199 opened by malin
#47200: Add ZipInfo.mode property
https://bugs.python.org/issue47200 opened by sam_ezeh
#47201: pip3.10.4 is configured with locations that require TLS/SSL, h
https://bugs.python.org/issue47201 opened by alessandro.pioli
#47204: Ensure PyEval_GetGlobals() doesn't set an exception when retur
https://bugs.python.org/issue47204 opened by ncoghlan
#47205: posix.sched_{get|set}affinity(-1) no longer returns ProcessLoo
https://bugs.python.org/issue47205 opened by christian.heimes
#47206: pickle docs are wrong about nested classes
https://bugs.python.org/issue47206 opened by JelleZijlstra
#47207: Switch datetime docstrings / documentation to using "Returns"
https://bugs.python.org/issue47207 opened by p-ganssle
#47208: Support libffi implementations that cannot support invocations
https://bugs.python.org/issue47208 opened by hoodchatham
#47209: Documentation for asserting values of `unittest.mock.Mock.call
https://bugs.python.org/issue47209 opened by himkt
#47214: builtin_function_or_method is also either a function or a meth
https://bugs.python.org/issue47214 opened by apostofes
#47215: Add "unstable" frame stack api
https://bugs.python.org/issue47215 opened by Mark.Shannon
#47216: adding mtime option to gzip open()
https://bugs.python.org/issue47216 opened by ellaellela
#47217: adding name to BZ2File
https://bugs.python.org/issue47217 opened by ellaellela
#47218: adding name to lzmafile
https://bugs.python.org/issue47218 opened by ellaellela
#47219: asyncio with two interpreter instances
https://bugs.python.org/issue47219 opened by mbadaire
#47220: Document the optional callback parameter of weakref.WeakMethod
https://bugs.python.org/issue47220 opened by maggyero
#47222: subprocess.Popen() should allow capturing output and sending i
https://bugs.python.org/issue47222 opened by pprindeville
#47228: Document that na??ve datetime objects represent local time
https://bugs.python.org/issue47228 opened by p-ganssle
#47231: TarFile.getmember cannot work on tar sourced directory over 10
https://bugs.python.org/issue47231 opened by cfernald
#47233: show_caches option affects code positions reported by dis.get_
https://bugs.python.org/issue47233 opened by 15r10nk
#47234: PEP-484 "numeric tower" approach makes it hard/impossible to s
https://bugs.python.org/issue47234 opened by tfish2
#47236: Document types.CodeType.replace() changes about co_exceptionta
https://bugs.python.org/issue47236 opened by vstinner
#47237: Inheritance from base class with property in class makes them
https://bugs.python.org/issue47237 opened by Germandrummer92
#47238: Python threading.Event().wait() depends on the system time
https://bugs.python.org/issue47238 opened by AleksandrAQ
#47240: Python 3.x built for ppc+ppc64 errs on: No module named 'msvcr
https://bugs.python.org/issue47240 opened by barracuda156
#47241: [C API] Move the PyCodeObject structure to the internal C API
https://bugs.python.org/issue47241 opened by vstinner
#47242: Annoying white bar in IDLE (line 457 in sidebar.py)
https://bugs.python.org/issue47242 opened by antudic
#47243: Duplicate entry in 'Objects/unicodetype_db.h'
https://bugs.python.org/issue47243 opened by LiarPrincess
#47244: email.utils.formataddr does not respect double spaces
https://bugs.python.org/issue47244 opened by AlecRosenbaum
#47247: Default arguments for access 'mode' parameters in pathlib and
https://bugs.python.org/issue47247 opened by AidanWoolley
#47248: Possible slowdown of regex searching in 3.11
[Python-Dev] Re: Declarative imports
You only get the ease-of-implementation benefit if you are willing to explicitly mark every _use_ of a lazy-imported name as special (and give the fully qualified name at every usage site). This is rather more cumbersome (assuming multiple uses in a module) than just explicitly marking an import as lazy in one location and then using the imported name in multiple places normally. Other "lazy import" solutions are trying to solve a problem where you want the name to be usable (without special syntax or marking) in many different places in a module, and visible in the module namespace always -- but not actually imported until someone accesses/uses it. The difficulty arises because in this case you need some kind of placeholder for the "deferred import", but you need to avoid this "deferred object" escaping and becoming visible to Python code without being resolved first. Explicitly marking which imports are lazy is fine if you want it (it's just a matter of syntax), but it doesn't do anything to solve the problem of allowing usage of the lazy-imported name to be transparent. I agree that the idea that top-of-module imports help readability is overstated; it sounds slightly Stockholm-syndrome-ish to me :) Top-of-module imports are frankly a pain to maintain and a pain to read (because they are often distant from the uses of the names). But they are a necessary evil if you want a) namespaces and b) not constantly retyping fully-qualified names at every usage site. Python is pretty committed to namespaces at this point (and I wouldn't want to change that), so that leaves the choice between top-of-module imports vs fully qualifying every use of every name; pick your poison. (Inline imports in a scope with multiple uses are a middle ground.) Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N4T2YMPHBLJXKCFA5CIPBFIZJJKO7SHR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 16:40, Guido van Rossum wrote: > The interesting idea here seems to make "lazy imports" easier to implement by > making them explicit in the code. So far, most lazy import frameworks for > Python have done hacks with `__getattribute__` overrides. IIRC the Cinder > version even modifies the bytecode and/or the interpreter. Disregarding the > specific notation proposed, *if* people would be willing to mark the points > where they expect lazy imports explicitly, that would make implementation > much simpler. Actually, to me the interesting idea is not so much lazy imports – I think they should not be lazy, at least that was my initial thought. I think they should be immediately resolved before anything else in that module: 1. This would settle any discussion about performance impact (there wouldn't be any). 2. This would enable IDEs, typers and other tooling to know the type using existing import logic. 3. Catch errors early! Notation is hard of course. Would users be willing to use it? I think so, at least in Rust, people do use it and I think just in the right places – typically for imports that are either "canonicalized" such as `re.compile` is in Python, or used just once. Cheers ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/NKBWGPJRKYQX4Z4WDDD74CSV4XEBGTFD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] The GitHub Issues Migration begins today
In about 1 hour from now, bugs.python.org will become read-only, and the migration to GitHub Issues will begin. Throughout the migration it will not be possible to report new issues or comment on existing ones. Once all the issues have been migrated from bpo to GitHub, you will be able to comment using GitHub Issues. bpo will remain available in read-only mode. For live updates, see https://discuss.python.org/t/github-issues-migration-status-update/14573 Best Regards, Ezio Melotti ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GUZI6Y7JTPMYY5BXAE56JGBER5VXOWEG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
The interesting idea here seems to make "lazy imports" easier to implement
by making them explicit in the code. So far, most lazy import frameworks
for Python have done hacks with `__getattribute__` overrides. IIRC the
Cinder version even modifies the bytecode and/or the interpreter.
Disregarding the specific notation proposed, *if* people would be willing
to mark the points where they expect lazy imports explicitly, that would
make implementation much simpler.
The argument that "imports on top" makes code more readable seems pretty
weak to me. The current hacks to speed up startup already violate this rule
(imports inside functions), and in most cases I start reading or writing
code in the middle of a file (having gotten there via a search in my
editor) and the meaning of an import is either obvious (e.g. re.match(...))
or requires another annoying search to find the definition of a certain
unknown variable. Tools can easily show all imports a module does.
The key questions to me are
- What should the notation be?
- Will users be willing to use it?
--Guido
On Fri, Apr 8, 2022 at 1:26 AM Malthe wrote:
> This is an idea which has been brought up before, sometimes introduced
> as "heresy". But an interesting twist has surfaced now which is
> typing.
>
> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports, usable anywhere using a simple
> syntax, @.
>
> For example, `some_regex = @re.compile(...)`.
>
> What happens then is that before anything else in that module, that
> symbol is imported:
>
> from re import compile as _mangled_re_compile
>
> It must be the very first thing (hoisting) because when else would it
> happen? It's been suggested before to have a shorthand syntax which
> does a dynamic import at the time of using it but this brings me to
> the twist:
>
> We want typing to pick up these imports. And this twist has a second
> leg which is that we often need to import symbols simply in order to
> type some argument type or return type. This leads to a great many
> more imports to type.
>
> (Nevermind that if you want to take typing further, abstract
> interfaces really make more sense rather than specific
> implementations, but the point is the same.)
>
> A situation where this would come in really handy is in scripting such
> as how we use Python in Apache Airflow to let users write out simple
> workflows. A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
>
> default_args = {
> "start_date": @datetime.datetime(...)
> }
>
> It's a lot more ergonomic from a user perspective (well perhaps for
> some users and for some programs).
>
> Thoughts?
>
> Cheers
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/UX6EJHLJNNLMFPWVPF5ANYHQSHDZK7SV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*
[Python-Dev] Re: Declarative imports
I do not think it's a "typing" improvement. I think what I like most in the proposal (which might be implemented differently - I am not too tied to this) is that it potentially brings huge optimisations in import times when currently a lot of imports are done unnecessarily early - slowing down startup times of tools that require parsing of python code. There are few examples of those: * Airflow DAGs - this is one of the problems we have in Airflow (and this is I think one of the reasons why Malthe brought it) is nicely described here https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#top-level-python-code * Tools like click-complete (where autocompletion suggestions are generated based on the decorators of Python code) * Static Python Typing checks (mostly when you declare Type returned/used by function that is only used inside the function). In all those cases, you really do not care about some of the imported code that is in some kind of "internal scope" and your python parsing does not **really** need to perform the imports to do the job. * In Airflow DAGs - tasks which are defined as functions (and default arguments from the example from Malthe) are not executed at all when the DAGs are parsed by scheduler. The DAGs are parsed to get the structure of the DAG. In this case we could optimise the time it take to parse the DAG and generate the structure. * In Click-complete - we only really care about names of the parameters for functions, but static typing for the function parmeters or returned value are not needed at all to generate complete suggestions. In this case importing time is essential, because you want to provide autocomplete suggestions **really quickly** * In case of typing of parameters or return values, it leads quite often to circular imports generated. If - in your module - the only place you use a type is inside one or two functions, adding static typing involves adding top-level import for that type. And that has huge potential of generating circular imports. Before static typing in parameters or return values of those, local imports would do the job nicely, but with typing added to parameters and return values, you cannot do local imports in those functions that have those parameters or return values. This often leads to adding local imports elsewhere, but in fact a "logical" thing is that those imports should be local for the functions and any of the scopes that the functions are called. Top-level imports in this case are causing unnecessary cross-module dependencies and circular imports. And some of those problems can currently be solved with local imports - but not all (default_values case by Malthe cannot). I think this idea has really good "need" behind. Maybe the way to implement could be improved on, but I see clearly the use cases and needs the "general idea" aims to solve. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ALXSM6753M7AT6MT7KVQULUCFX5G4XT7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On 4/8/22 09:34, Petr Viktorin wrote: > some cases where Discourse thinks something is a footer and removes it, but > IMO they're not huge problems. It also has some very good markdown support, so you can post nicely formatted code via email. Mailing list mode with Discourse is almost nicer than a plain email list in some ways. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5CP57WIW3IO2FF2L4Y4EU3RBY5FS2DAB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
While it's different than you proposal, some people may like this magic "smart imports" project which lazily imports a module when it's used :-) Project: https://pypi.org/project/smart-imports/ Replace: --- import math from my_project import calc # 100500 other imports def my_code(argument, function=calc): return math.log(function(argument)) --- with: --- import smart_imports smart_imports.all() # no any other imports def my_code(argument, function=calc): return math.log(function(argument)) --- I don't like it :-) I prefer explicit imports at the top! Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KLXXAFVPY6RMGJVIFPJ5UNIXNO2YG472/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Heads-up: Converting Misc/stable_abi.txt to TOML
Hello, Now that tomllib is in the stdlib, I'd like to convert the stable ABI manifest (Misc/stable_abi.txt) to TOML to make it easier to work with. To get proper highlighting in editors, I'd like to rename it from .txt to .toml. So, when the command that uses it stops working for you, change `.txt` to `.toml`. (See https://devguide.python.org/c-api/#adding-a-new-definition-to-the-limited-api -- if you're using make, nothing should change for you.) As always, the file is only intended as input to the stable_abi script. The format/contents can change unexpectedly. If you use it for something else, let me know and let's create a proper data source for your use case. bpo link: https://bugs.python.org/issue47168 ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C66LFKFO23BACJ673FVF7E2VS5TBP6DW/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On 4/8/2022 12:51 PM, Daniel Pope wrote: On Fri, 8 Apr 2022 at 12:23, Steve Dower wrote: I've read the rest of the thread so far, and agree strongly that we can't do this at the language/runtime level. You mean the hoisting, right? I don't see any reason why an import expression without hoisting would be impractical. But I'd like to hear your thoughts if you think it is. Sure, __import__() or better yet, importlib.util.import_module() are perfectly good ways to do an import in an expression. Both have exactly the right amount of code-smell as well, and only a few surprising ways to interfere with the behaviour of apparently completely unrelated code ;) As others pointed out, implicitly defining variables that outlive the evaluation of an expression is incredibly messy. It was bad enough getting assignment expressions defined well enough to (a) not be a terrible footgun and (b) be possible for the average developer to predict what will happen when they're used (arguably this isn't true either, but the "average" developer probably isn't doing things that will hit the edge cases). We don't want to have to figure something like that out again. Certainly not for the sake of typing - the SC has already ruled that language development won't be driven by the convenience of static type users. Cheers, Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SRAIPD7DQH36DNBZVEAA6I56IZHLGA66/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On 08. 04. 22 13:40, Greg Ewing wrote: On 8/04/22 12:13 pm, Gregory P. Smith wrote: And for lurkers and subscribers here to enable email notifications for categories of interest over there. Is it possible to participate in a Discourse discussion entirely by email, without having to visit the web site? It's possible. I follow Discourse by e-mail, in the "Mailing list mode", where Discourse sends me all comments and I filter in the e-mail client. That works rather well for me. Personally I visit the web app to reply, but it's possible to reply by e-mail. There are some issues with formatting, and some cases where Discourse thinks something is a footer and removes it, but IMO they're not huge problems. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/REO5BCKR64EB4T5P3ZDT2VYSTFKNDV2G/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On Apr 8, 2022, 6:49 AM -0500, Greg Ewing , wrote: > On 8/04/22 12:13 pm, Gregory P. Smith wrote: > > And > > for lurkers and subscribers here to enable email notifications for > > categories of interest over there. > > Is it possible to participate in a Discourse discussion entirely > by email, without having to visit the web site? > It should be possible to fully interact with Discourse from an email client, Mozilla has some nice guidance on this: https://discourse.mozilla.org/t/how-do-i-use-discourse-via-email/15279 > If not, then if python-dev becomes discourse-only, I will no > longer be able to follow or participate in it. I don't have the > time or energy to chase around visiting a bunch of clunky web > interfaces every day. > > -- > Greg > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/ANKS4G2QXS5FJ2UZJ52RZWVFXHT76QG7/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/UM5GPP3XTJJ4DCNYJ2KOSVQL3ETPZJCH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 12:57, Daniel Pope wrote:
>
> On Fri, 8 Apr 2022 at 12:23, Steve Dower wrote:
> >
> > I've read the rest of the thread so far, and agree strongly that we
> > can't do this at the language/runtime level.
>
> You mean the hoisting, right?
>
> I don't see any reason why an import expression without hoisting would
> be impractical. But I'd like to hear your thoughts if you think it is.
>
> Desirability on the other hand is subjective. I think I actually do
> desire it, others are not bothered. I don't see strong arguments as to
> why you definitely wouldn't want it in your language.
OK, I'll be explicit. I don't want this in Python. Having imports at
the top of a file makes it much easier to see your dependencies. The
current language features, combined with community wide style guides
that discourage the use of local imports, make this straightforward,
while still providing means of doing local imports if needed.
This proposal isn't just about having a new syntax to do an on-demand
import. It's also about normalising the idea that people can pull code
from other modules without declaring the intent to do so up front.
While I don't dispute that in some circumstances (notably the REPL,
and throwaway scripts[1]) not having to add import statements would be
nice, I don't see how we'd limit usage to cases like that - and I
*really* don't want to have to work out where some weird inline import
happened while debugging a priority 1 bug in a 10,000 line code base
at 2am...
Also, it should be possible to do something like this using the
existing import machinery:
from magic_import import on_demand_loader as OD
...
# Many lines of code later
default_args = {
"start_date": OD.datetime.datetime(...)
}
Trivial proof of concept implementation:
class OD:
def __getattr__(self, name):
return __import__(name)
OD = OD()
print(OD.datetime.datetime.now())
Paul
[1] Of course, today's throwaway script usually turns out to be next
month's key component in a mission-critical software stack :-(
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/HC4UJPBWJHU5JWIPQQKNJRMN2QBXHJVC/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 12:23, Steve Dower wrote: > > I've read the rest of the thread so far, and agree strongly that we > can't do this at the language/runtime level. You mean the hoisting, right? I don't see any reason why an import expression without hoisting would be impractical. But I'd like to hear your thoughts if you think it is. Desirability on the other hand is subjective. I think I actually do desire it, others are not bothered. I don't see strong arguments as to why you definitely wouldn't want it in your language. Design is hard, but designing this is definitely not as hard as designing match/case or except* statements. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/G4TOKDQPJPKNEGPUGYMJFOHE7FSID2OY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On 8/04/22 12:13 pm, Gregory P. Smith wrote: And for lurkers and subscribers here to enable email notifications for categories of interest over there. Is it possible to participate in a Discourse discussion entirely by email, without having to visit the web site? If not, then if python-dev becomes discourse-only, I will no longer be able to follow or participate in it. I don't have the time or energy to chase around visiting a bunch of clunky web interfaces every day. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ANKS4G2QXS5FJ2UZJ52RZWVFXHT76QG7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 11:26, Malthe wrote: > Perhaps `some_regex = re::compile(r"...")` could work. > > That is, :: to delineate the import. As I mentioned in the Discourse thread, this syntax chimes with Rust (and C++, and also PHP, triggering fond memories of Paamayim Nekudotayim). There are two problems, sort of on the minor side: First, there's one place in Python where this can occur and that is in slices: xs[start::step] means the slice from start to the end with a step of step. But xs[(module::name)] is currently a syntax error, so you could just require parentheses to disambiguate the case. Still, it is confusing if you write something slightly more complex like xs[1 + math::factorial(n)] and you get NameError: name 'math' is not defined. Secondly, it precludes importing just a module object, you have to import a name from a module. Maybe using a module object in an expression is niche and can be disregarded. Or a solution could be to allow an expression like (re::) to import a module object, which I guess we could get used to. Also fun would be to consider what ::name does - access to current globals without the global keyword? I think this discussion is better suited to the python-ideas mailing list, or the Discourse thread I linked earlier. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MO524TQHUR7PMUWOZ4PBQAXDMOOF2KFP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
I've read the rest of the thread so far, and agree strongly that we can't do this at the language/runtime level. However ... On 4/8/2022 9:24 AM, Malthe wrote: This is an idea which has been brought up before, sometimes introduced as "heresy". But an interesting twist has surfaced now which is typing. [SNIP] We want typing to pick up these imports. And this twist has a second leg which is that we often need to import symbols simply in order to type some argument type or return type. This leads to a great many more imports to type. Type checkers can do whatever they like. The syntax has to remain a valid Python expression, but the semantics of it can include implicit imports if that's what you want it to do. I'd certainly make use of it on those occasions I actually put type annotations in code (rather than .pyi files). But it doesn't need the runtime to do anything differently if the module isn't really going to be used at that point. Cheers, Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/57NDZA7B3Z4PLGS5JON5YXOP44KI4LVU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 10:27, Malthe wrote:
>
> On Fri, 8 Apr 2022 at 08:51, Paul Moore wrote:
> > Are you exaggerating for effect here or would this *actually* just expand to
> >
> > from datetime import datetime
> > default_args = {
> > "start_date": datetime(...)
> > }
>
> Yes – and of course that is just a snippet, an actual complete script
> will have lots of such imports.
>
> The point is that in a scripting situation (especially one in which
> you have many small scripts) the
> top-level import requirement for simple imports like `datetime`
> becomes rather verbose.
But Python's origin is in scripting situations, and explicit imports
for scripts has always been both the normal approach, and one of
Python's *strengths* ("Explicit is better than implicit"). Arguing
that explicit imports are a bad thing in (general) scripts is arguing
against decades of history and experience. If there's a reason why
*your specific context* would benefit from an abbreviated form, you
need to present it. But arguing that explicit imports are too verbose
for all cases of scripting isn't going to fly, frankly.
Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/TMJ4ZI7J62C64PZXQU3WRLFA6FAKPRVB/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 10:15, Daniel Pope wrote: > But, your proposed syntax is not usable because it is ambiguous. Exactly what > you propose is already used for decorators in a way that the parser would not > be able to distinguish a decorator from an import expression. Consider: > > @ham.spam() # import expression or decorator? > > def eggs(): > ... > > This currently parses as a decorator and for backwards compatibility that > must not change, which means that import expressions would be usable in some > contexts and not others purely based on what follows them. Perhaps `some_regex = re::compile(r"...")` could work. That is, :: to delineate the import. @breakfast.ham::spam def eggs(): pass Cheers ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VXOSHFQDLPI5KXVNXWKFMRZOZHYJLZL5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022, 09:30 Malthe, wrote: > For example, `some_regex = @re.compile(...)`. > I like the idea of import expressions. I pitched it on Discourse recently: https://discuss.python.org/t/import-expressions/11582 However, I do not see hoisting as something that should be done by Python. In that thread it was suggested your IDE could do it, or maybe isort, which seems fine because it's at the programmer's discretion and ends up being explicit. But, your proposed syntax is not usable because it is ambiguous. Exactly what you propose is already used for decorators in a way that the parser would not be able to distinguish a decorator from an import expression. Consider: @ham.spam() # import expression or decorator? def eggs(): ... This currently parses as a decorator and for backwards compatibility that must not change, which means that import expressions would be usable in some contexts and not others purely based on what follows them. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IS7WHO3ZL33D4PSHWFSLU4D4WQ5736MY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, Apr 08, 2022 at 08:24:40AM +, Malthe wrote:
> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports,
I'm not sure I understand why you consider this "declarative".
https://en.wikipedia.org/wiki/Declarative_programming
As I see it, this proposal merely adds a unary operator for implicit
imports.
> For example, `some_regex = @re.compile(...)`.
>
> What happens then is that before anything else in that module, that
> symbol is imported:
>
> from re import compile as _mangled_re_compile
>
> It must be the very first thing (hoisting) because when else would it
> happen?
On-demand. As you say yourself, this could be a dynamic import at the
time of use.
> It's been suggested before to have a shorthand syntax which
> does a dynamic import at the time of using it but this brings me to
> the twist:
>
> We want typing to pick up these imports.
At the moment static type checkers have to look for two import
statements: `import spam`, and `from spam import eggs`.
With this proposal, they will still need to look for those two import
statements, but they will also need to parse every expression looking
for `@` as a unary operator:
result = 2*x**3 - 3*y + @math.sin(x*y) - 5*x*y**2
This does not help typing at all. It just means the type-checker has to
do more work to recognise imports.
> And this twist has a second
> leg which is that we often need to import symbols simply in order to
> type some argument type or return type. This leads to a great many
> more imports to type.
Code is read much more than it is written. I would much prefer to have
explicit imports (by convention, if not necessity) in one place at the
top of the page, than to mystery symbols where the implicit import could
be buried far away.
Right now, this is an error:
# Start of module.
obj = wibble.foo
because wibble is not a built-in nor a global (excluding weird
shenanigans committed by other modules), so the name "wibble" doesn't
exist. But with hoisting, that import could be *anywhere* in the file.
Even in dead code.
# Start of module.
obj = wibble.foo
...
...
# five pages of code later
for a in obj.method():
while flag:
if condition:
@wibble
That's right, with hoisting you can use a module before you import it.
Mind. Blown.
Have pity on beginners, casual Python programmers, and the people who
have to help them. Don't implement this horror.
If it were merely on-demand imports, then we would know that the import
@wibble would have to appear in actual, executed code before you can use
wibble. But with hoisting, we don't even have that promise.
It is truly spooky action at a distance. And to save nothing but an
import line.
> A situation where this would come in really handy is in scripting such
> as how we use Python in Apache Airflow to let users write out simple
> workflows. A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
>
> default_args = {
> "start_date": @datetime.datetime(...)
> }
That's just a three-liner, which becomes a four-liner:
import datetime
default_args = {
"start_date": datetime.datetime(...)
}
--
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/GILL25XYSAPF4FN7LTZC7XLDB7ZX4E4Y/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API
On 07. 04. 22 17:10, Victor Stinner wrote:
On Thu, Apr 7, 2022 at 12:02 PM Petr Viktorin wrote:
- This API stays with the regular public API (Include/cpython/), but to
use it you'll need to #define Py_USING_UNSTABLE_API (name up for
bikeshedding).
Since there is already something similar called "Py_LIMITED", I
suggest dropping "USING_" for just: "Py_UNSTABLE_API".
But I really like the explicit “USING” :)
I don't think there's too much value in these two being consistent.
(Yay, bikeshedding!)
- The functions will be renamed to drop the leading underscore. The old
names will be available as aliases (using #define) and may be removed
whenever the API changes. (Ideally, the underscore should always mark
API that's fully private with no guarantees at all.)
Should functions entering the "unstable API" be documented and tested?
For example, _PyEval_EvalFrameDefault() and
_PyInterpreterState_SetEvalFrameFunc() have no test nor doc.
Yes.
I'll add docs, for tests I'll at least open an issue.
This applies to:
- PyCode_New, PyCode_NewWithPosOnlyArgs
It would be nice to update Cython to define the Py_UNSTABLE_API macro
before the macro is required to get the function, since Cython still
uses PyCode_New().
That (for any project, not just Cython) is why I propose warnings in
3.11, before requiring the opt-in in 3.12.
Should we deprecate types.CodeType constructor in the Python API,
since types.CodeType.replace() exists and seems to be a better API
("more stable")?
I don't know. But it's for a different discussion.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/MBDJH4J5SA2GJENCBWTHA53KURPX2S4J/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On 08. 04. 22 1:26, Ethan Furman wrote: On 4/7/22 07:31, Petr Viktorin wrote: On 07. 04. 22 15:59, Victor Stinner wrote: Would it be possible to announce new PEPs on python-dev please? Currently, all PEPs should be announced on python-dev, but not necessarily right after they're published. They should be announced before submitting them to SC, though. By the time a PEP is being submitted, the opportunity to contribute has passed. That's a goal the PEP author should work toward, by submitting a PEP after all relevant discussion took place. The SC's job is to check this, as well as it can, and ask for more discussion if needed. If you think you haven't been included in the conversation, it's never too late to make your point. Of course, the discussion can take place in a forum you don't monitor. (And that's a good thing -- as an extreme example, we probably don't want to have typing-sig or packaging discussions here.) But every PEP currently needs to be at least announced on python-dev, so everyone should at least know there is a PEP to discuss. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ND7UUICXJFR5CY76W5CIDXRQTEXZMGOR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 08:51, Paul Moore wrote:
> Are you exaggerating for effect here or would this *actually* just expand to
>
> from datetime import datetime
> default_args = {
> "start_date": datetime(...)
> }
Yes – and of course that is just a snippet, an actual complete script
will have lots of such imports.
The point is that in a scripting situation (especially one in which
you have many small scripts) the
top-level import requirement for simple imports like `datetime`
becomes rather verbose.
Thanks
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/FVYRPSN6A7KN73MTQIF2UIU5W3HSUPGV/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
Hi, Le 08/04/2022 à 03:52, David Mertz, Ph.D. a écrit : > FWIW, I find Discourse (and everything similar that I've seen), awkward, > difficult to use, poorly organized, and in every way inferior to my mail > client. Another Discourse misfeature I just came across: history rewriting. I know it's by design, but, at least for casual readers, it makes it hard to know which information is new. Concrete case: the GitHub Issues transition post ( https://discuss.python.org/t/github-issues-migration-is-coming-soon/13791 ) is displayed as a "read" link in my browser for more than a month. Still, I just found out that it was edited twice in between. How am I supposed to know? Just as a data point, Baptiste ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/53QL5TFMIDFUJ5U2XEXVN2543VEXJY6E/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 09:29, Malthe wrote:
> A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
>
> default_args = {
> "start_date": @datetime.datetime(...)
> }
Are you exaggerating for effect here or would this *actually* just expand to
from datetime import datetime
default_args = {
"start_date": datetime(...)
}
I'm not suggesting that there isn't a genuine benefit here, but your
example doesn't demonstrate what it is.
Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/RRLFDL2BDF3UV646GVZVRXUSGM7X5FRQ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Declarative imports
On Fri, 8 Apr 2022 at 18:29, Malthe wrote: > > This is an idea which has been brought up before, sometimes introduced > as "heresy". But an interesting twist has surfaced now which is > typing. > > But firstly, let me present the idea. It is very simple, that Python > should have declarative imports, usable anywhere using a simple > syntax, @. > > For example, `some_regex = @re.compile(...)`. > > What happens then is that before anything else in that module, that > symbol is imported: > > from re import compile as _mangled_re_compile > > It must be the very first thing (hoisting) because when else would it > happen? JavaScript has this feature. ES6 modules are always imported declaratively and always before the rest of the script is executed. It is *extremely* annoying. Most of the time, it behaves the same way as Python's dynamic imports, but any time it doesn't, the "imports before the rest of the code" feature is nothing but hassle. Here's an alternative: Have a whitelist of modules that you make use of in this way. Any time you refer to a name that's on that whitelist and doesn't have an assigned meaning in the module, insert an import statement at the top of the script. It's not too hard to set up your editor to be able to do this (eg in SciTE, I can press Ctrl+1 to quickly run a script; a lot of editors let you configure a code formatter that runs automatically on save, which would also work), and then you retain full control of exactly where the importing happens. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YUNFSHIONZNLZ6QS3GWF4QVLP4IWZY6W/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Declarative imports
This is an idea which has been brought up before, sometimes introduced
as "heresy". But an interesting twist has surfaced now which is
typing.
But firstly, let me present the idea. It is very simple, that Python
should have declarative imports, usable anywhere using a simple
syntax, @.
For example, `some_regex = @re.compile(...)`.
What happens then is that before anything else in that module, that
symbol is imported:
from re import compile as _mangled_re_compile
It must be the very first thing (hoisting) because when else would it
happen? It's been suggested before to have a shorthand syntax which
does a dynamic import at the time of using it but this brings me to
the twist:
We want typing to pick up these imports. And this twist has a second
leg which is that we often need to import symbols simply in order to
type some argument type or return type. This leads to a great many
more imports to type.
(Nevermind that if you want to take typing further, abstract
interfaces really make more sense rather than specific
implementations, but the point is the same.)
A situation where this would come in really handy is in scripting such
as how we use Python in Apache Airflow to let users write out simple
workflows. A workflow definition which could be a 5-liner quickly
becomes a 20-liner – consider for example:
default_args = {
"start_date": @datetime.datetime(...)
}
It's a lot more ergonomic from a user perspective (well perhaps for
some users and for some programs).
Thoughts?
Cheers
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/UX6EJHLJNNLMFPWVPF5ANYHQSHDZK7SV/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: [python-committers] Re: [IMPORTANT] Preparations for 3.11.0 beta 1
Thank you, Victor. I had considered dropping (a) from the PEP. But I keep them because: * I rushed to write PEP, before 3.11 beta. * In the "Backward compatibility" section in the PEP, I want to mention `locale.getencoding()` and `encoding="locale"` * But they are not fixed in the main branch yet. So I need to include what needs to be fixed in 3.11 in the PEP. But for now, we are close to merge `locale.getencoding()`. And I am afraid merging it before the PEP accepted even though it is documented in the PEP... Now I think the best way is: * Withdraw the PEP submission temporarily. * Implement `locale.getencoding()` and fix `encoding="locale"` in the main branch. * Remove them from the PEP. * Resubmit the PEP. And if the PEP is accepted, I want to do this in the 3.11 branch (even though it will be beta already): * Improve document about UTF-8 mode and EncodingWarning based on the PEP. * Add (opt-in) EncodingWarning to `locale.getpreferredencoding()` and `subprocess.Popen(text=True)`. On Thu, Apr 7, 2022 at 9:42 PM Victor Stinner wrote: > > IMO adding locale.getencoding() to Python 3.11 is not controversial > and is useful even if PEP 686 is rejected. This function was discussed > for 1 year (bpo-43510, bpo-43552, bpo-43557, bpo-47000) and there is > an agreement that there is a need for this function. > > > Making `open(path, encoding="locale")` use locale encoding in UTF-8 mode > > (Python 3.10 used UTF-8) > > If someone explicitly opts in for the "locale encoding", it sounds > surprising that the locale (encoding) is ignored and that UTF-8 is > used if the Python UTF-8 Mode is enabled. I'm fine with this change. > If you want to always UTF-8... Pass explicitly UTF-8: > > # no surprise, always decode file content from UTF-8 > json_file = open(filename, encoding="utf-8") > > -- > > I will not comment PEP 686 here. It's being discussed on Discourse: > > * https://discuss.python.org/t/14435 > * https://discuss.python.org/t/14737 > > Victor > > On Thu, Apr 7, 2022 at 5:35 AM Inada Naoki wrote: > > > > Hi, Pablo. > > > > I just submitted the PEP 686 to the SC. > > https://github.com/python/steering-council/issues/118 > > > > In this PEP, I am proposing: > > > > a. Small improvement for UTF-8 mode in Python 3.11 > > b. Make UTF-8 mode default in Python 3.13. > > > > (a) is an important change for (b) so I included it in the PEP. > > More precisely, (a) contains two changes: > > > > * Making `open(path, encoding="locale")` use locale encoding in UTF-8 > > mode (Python 3.10 used UTF-8) > > * Add `locale.getencoding()` that is same to > > `locale.getpreferredencoding(False)` but returns locale encoding even > > in UTF-8 mode. > > > > These changes are important for (b). > > But they are not a big change needing PEP. > > > > What should I do? > > > > * Do not merge anything until PEP accepted. > > * Merge (a) without waiting PEP accepted. > > * Merge (a) and remove it from the PEP. > > > > FWI, I and Victor are implementing `locale.getencoding()` for now. > > > > https://bugs.python.org/issue47000 > > https://github.com/python/cpython/pull/32068 > > > > Regards, > > -- > > Inada Naoki > > ___ > > python-committers mailing list -- python-committ...@python.org > > To unsubscribe send an email to python-committers-le...@python.org > > https://mail.python.org/mailman3/lists/python-committers.python.org/ > > Message archived at > > https://mail.python.org/archives/list/python-committ...@python.org/message/7E4QEKZ6HNDDPDL76LP3TBBKLAUQ7AHB/ > > Code of Conduct: https://www.python.org/psf/codeofconduct/ > > > > -- > Night gathers, and now my watch begins. It shall not end until my death. -- Inada Naoki ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XVJEAF7S2OORL77QMLLQTWKHLRDFA3KH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: About PEPs being discussed on Discourse
On Thu, Apr 7, 2022 at 7:19 PM David Mertz, Ph.D. wrote: > FWIW, I find Discourse (and everything similar that I've seen), awkward, > difficult to use, poorly organized, and in every way inferior to my mail > client. > > I don't think I'm entirely alone in this experience though. > You're not -- I agree This was brought up when Discourse was first introduced (as an experiment) --and I suspect both you and I made a similar comment. But if Discourse has been adopted, I guess it's time for us curmudgeons to bite the bullet and start monitoring it -- and frankly, this list (and python-ideas) should probably be retired, or turned into an announcement-only list -- having the current split is the worst option of all. -CHB -- Christopher Barker, PhD (Chris) Python Language Consulting - Teaching - Scientific Software Development - Desktop GUI and Web Development - wxPython, numpy, scipy, Cython ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/C23QR5T65VHNL4ZF7CJYJQGEZ4LDLS6S/ Code of Conduct: http://python.org/psf/codeofconduct/
