[Python-Dev] Re: Declarative imports

[Christopher Barker]

On Mon, Apr 11, 2022 at 2:03 PM Brett Cannon  wrote:

> That is an extremely subtle shift for what `import x.y` does compared to
> `(import x.y)`. That requires a context switch of not only seeing `import`
> in an expression context, but that the statement also acts differently in
> terms of what is returned by the equivalent statement. I really don't to
> try and teach that distinction to a newcomer.
>

Frankly, there are already an overwhelming-to-a-newcomer number of ways to
import modules. We really don't want nore!

-CHB


-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FR3SSOXKWPQJNI2YGLITOW6NXADZGMCU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Brett Cannon]

On Sun, Apr 10, 2022 at 2:39 AM Daniel Pope  wrote:

> On Fri, 8 Apr 2022, 17:44 Guido van Rossum,  wrote:
>
>> The interesting idea here seems to make "lazy imports" easier to
>> implement by making them explicit in the code. So far, most lazy import
>> frameworks for Python have done hacks with `__getattribute__` overrides.
>>
>
> The value is more than ease of implementation. Having syntax for import
> expressions makes them statically analysable, which is needed for type
> checkers and IDE autocompletion.
>
> But I also see value in being able to type out code that uses modules not
> yet imported without breaking my flow to add an import statement. I don't
> yet trust IDEs to do this because I've been bitten by them doing so
> incorrectly in the past.
>
> The key questions to me are
>> - What should the notation be?
>>
>
> I would like to bid again for (import package.module) as an expression.
> Instead of doing the import and assigning package to a variable package it
> would evaluate to the module object package.module.
>

That is an extremely subtle shift for what `import x.y` does compared to
`(import x.y)`. That requires a context switch of not only seeing `import`
in an expression context, but that the statement also acts differently in
terms of what is returned by the equivalent statement. I really don't to
try and teach that distinction to a newcomer. And I don't think the
ergonomics are great enough to warrant the context switch.

Plus you can do this today with imortlib.import_module(). Unless you're
suggesting the name also be bound in the scope it's run in? In which case
that's `(abc := importlib.import_module("collections.abc")).Mapping`.

-Brett


>
> The `as` form is not needed because no name is assigned and the `from`
> form isn't as valuable because you can just use attribute access afterwards.
>
> It isn't terse but it does make use of the import keyword and is thus
> instantly recognisable. It is even syntax highlighted correctly by much
> existing software. If we're using the import keyword then I think it has to
> look like this.
>
> But I concede that it isn't particularly elegant to type hint things with
>
> (import collections.abc).Mapping
>
> ...but not so inelegant that I couldn't see myself using it for a few
> one-off imports per module.
>
> A quirk is that it means there's a big difference between the statements
>
> import foo
>
> and
>
> (import foo)
>
> because one assigns a variable. I don't mind that; I don't think it is too
> ambiguous to a reader.
>
>
>
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/T6V7WZAFEXGWMHXLHS7XHYXI5OPMOZKA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5HMNON4YODBJNHSSAYT4B7CRZJUKFGW4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Barry Warsaw]

Thanks Carl,

> On Apr 9, 2022, at 08:25, Carl Meyer  wrote:
> 
> Our experience in practice, though, has been that universally lazy
> imports is somewhat easier to adopt than Strict Modules, and has had a
> much bigger overall impact on reducing startup time for big CLIs (and
> a big web server too; as you note it's not as serious an issue for a
> web server in production, but restart time still does make a
> difference to dev speed / experience.)

Excellent point about the impact of restarts and development time.  That’s been 
an issue for us a bit, but not an overwhelming motivation to rewrite in other 
languages[1].

> Removing slow stuff happening
> at import time helps, but it'll never match the speed of not doing the
> import at all! We've seen startup time improvements up to 70% in
> real-world CLIs just by making imports lazy. We've also opened an
> issue to discuss the possibility of upstreaming this. [2]
> 
> [1] https://github.com/facebookincubator/cinder/#strict-modules
> [2] https://bugs.python.org/issue46963

Post-GH-issues-migration link for the issue: 
https://github.com/python/cpython/issues/91119

I’ve put some questions and comments there, but I’m also really curious about 
the technical details for your lazy imports.  Have you gotten as far as 
thinking about a PR or PEP?

-Barry

[1] Not that there aren’t other reasons folks give for rewriting, such as 
multicore performance, ecosystem alignment (e.g. SREs being more comfortable in 
Go), etc.



signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/47BLABCM4G3DQ5WE5MDGY2ISOHHTTUHO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Patrick Reader]

On 10/04/2022 15:52, Guido van Rossum wrote:

On Sun, Apr 10, 2022 at 2:31 AM Daniel Pope  wrote:

On Fri, 8 Apr 2022, 17:44 Guido van Rossum,  wrote:

The interesting idea here seems to make "lazy imports" easier
to implement by making them explicit in the code. So far, most
lazy import frameworks for Python have done hacks with
`__getattribute__` overrides.


The value is more than ease of implementation. Having syntax for
import expressions makes them statically analysable, which is
needed for type checkers and IDE autocompletion.


This has been brought up a few times and I don't get it. Currently a 
use of an imported module is perfectly analyzable by all the static 
type checkers I know of (e.g. mypy, Pyre, pyright). For the static 
analyzer it makes no difference if I have


import re
.
.
.
def foo(x):
    if re.match(r"blah", x): ...

or the hypothetical inline form:

def foo(x):
    if @re.match(r"blah", x): ...

But I also see value in being able to type out code that uses
modules not yet imported without breaking my flow to add an import
statement. I don't yet trust IDEs to do this because I've been
bitten by them doing so incorrectly in the past.


I have too.


I see no reason why @re.match(...) couldn't just be equivalent to 
__import__("re").match(...) (or a more optimised compiled version of 
that). Any side-effects of importing the module (of which there probably 
shouldn't be any for most modules) just happen at whenever the first 
time the @re.match(...) is evaluated.


After the first call, and unless __import__ is overloaded, I also don't 
think @re.match(...) would have to be significantly slower than 
re.match(...). The latter already has to look up a global variable 
(assuming the module was imported globally, which is almost always the 
case). I don't think it would be too hard to optimise and Quicken.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3P5KUDUZXOIEZSEWNV7UMVAQNGYPGXDF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Sasha Kacanski]

To me Steve nailed it... static type checker = lean and mean...

On Fri, Apr 8, 2022, 5:03 AM Steven D'Aprano  wrote:

> On Fri, Apr 08, 2022 at 08:24:40AM +, Malthe wrote:
>
> > But firstly, let me present the idea. It is very simple, that Python
> > should have declarative imports,
>
> I'm not sure I understand why you consider this "declarative".
>
> https://en.wikipedia.org/wiki/Declarative_programming
>
> As I see it, this proposal merely adds a unary operator for implicit
> imports.
>
>
>
> > For example, `some_regex = @re.compile(...)`.
> >
> > What happens then is that before anything else in that module, that
> > symbol is imported:
> >
> > from re import compile as _mangled_re_compile
> >
> > It must be the very first thing (hoisting) because when else would it
> > happen?
>
> On-demand. As you say yourself, this could be a dynamic import at the
> time of use.
>
>
> > It's been suggested before to have a shorthand syntax which
> > does a dynamic import at the time of using it but this brings me to
> > the twist:
> >
> > We want typing to pick up these imports.
>
> At the moment static type checkers have to look for two import
> statements: `import spam`, and `from spam import eggs`.
>
>
> With this proposal, they will still need to look for those two import
> statements, but they will also need to parse every expression looking
> for `@` as a unary operator:
>
> result = 2*x**3 - 3*y + @math.sin(x*y) - 5*x*y**2
>
> This does not help typing at all. It just means the type-checker has to
> do more work to recognise imports.
>
>
>
> > And this twist has a second
> > leg which is that we often need to import symbols simply in order to
> > type some argument type or return type. This leads to a great many
> > more imports to type.
>
> Code is read much more than it is written. I would much prefer to have
> explicit imports (by convention, if not necessity) in one place at the
> top of the page, than to mystery symbols where the implicit import could
> be buried far away.
>
> Right now, this is an error:
>
> # Start of module.
> obj = wibble.foo
>
> because wibble is not a built-in nor a global (excluding weird
> shenanigans committed by other modules), so the name "wibble" doesn't
> exist. But with hoisting, that import could be *anywhere* in the file.
> Even in dead code.
>
> # Start of module.
> obj = wibble.foo
> ...
> ...
> # five pages of code later
> for a in obj.method():
> while flag:
> if condition:
> @wibble
>
> That's right, with hoisting you can use a module before you import it.
> Mind. Blown.
>
> Have pity on beginners, casual Python programmers, and the people who
> have to help them. Don't implement this horror.
>
> If it were merely on-demand imports, then we would know that the import
> @wibble would have to appear in actual, executed code before you can use
> wibble. But with hoisting, we don't even have that promise.
>
> It is truly spooky action at a distance. And to save nothing but an
> import line.
>
>
> > A situation where this would come in really handy is in scripting such
> > as how we use Python in Apache Airflow to let users write out simple
> > workflows. A workflow definition which could be a 5-liner quickly
> > becomes a 20-liner – consider for example:
> >
> > default_args = {
> > "start_date": @datetime.datetime(...)
> > }
>
> That's just a three-liner, which becomes a four-liner:
>
> import datetime
> default_args = {
> "start_date": datetime.datetime(...)
> }
>
>
>
> --
> Steve
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/GILL25XYSAPF4FN7LTZC7XLDB7ZX4E4Y/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PVOJ74BCX6YA5L5ZCIHGXT6STKRNK4AI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Malthe]

On Sun, 10 Apr 2022 at 09:31, Daniel Pope  wrote:
> I would like to bid again for (import package.module) as an expression. 
> Instead of doing the import and assigning package to a variable package it 
> would evaluate to the module object package.module.

I like this proposal and I agree with previous posters about hoisting
being undesirable.

And if lazy importing is slow then that is probably something that can
be optimized away; of course in tight loops or performance-sensitive
code you wouldn't use this functionality.

But perhaps using "from" makes the behavior more clear (and it is shorter):

start_date = from datetime.datetime(2012, 1, 6)

I think the behavior could be to simply keep importing until we have a
non-module value, rather than rely on putting the import name in
parentheses.

Cheers
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2NLRKDG3TZCQSALAJVHOM3VKZH45ZYWH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Daniel Pope]

On Sun, 10 Apr 2022, 15:53 Guido van Rossum,  wrote:

> On Sun, Apr 10, 2022 at 2:31 AM Daniel Pope  wrote:
>
>> On Fri, 8 Apr 2022, 17:44 Guido van Rossum,  wrote:
>>
>>> The interesting idea here seems to make "lazy imports" easier to
>>> implement by making them explicit in the code. So far, most lazy import
>>> frameworks for Python have done hacks with `__getattribute__` overrides.
>>>
>>
>> The value is more than ease of implementation. Having syntax for import
>> expressions makes them statically analysable, which is needed for type
>> checkers and IDE autocompletion.
>>
>
> This has been brought up a few times and I don't get it. Currently a use
> of an imported module is perfectly analyzable by all the static type
> checkers I know of (e.g. mypy, Pyre, pyright).
>

I was comparing a hypothetical import expression syntax with alternatives
like __getattribute__ hacks, which I take to mean
somemagicobject.package.module.attribute, or as other have suggested,
importlib.import_module(). I believe those are not statically analysable at
least without special casing them in the type checker.

An import expression would be just as statically analysable as a statement,
while being more ergonomic in certain situations.

>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2UXMBKQJHN5AM3FKDUZ4DEF7TPD4HGYP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Guido van Rossum]

On Sun, Apr 10, 2022 at 2:31 AM Daniel Pope  wrote:

> On Fri, 8 Apr 2022, 17:44 Guido van Rossum,  wrote:
>
>> The interesting idea here seems to make "lazy imports" easier to
>> implement by making them explicit in the code. So far, most lazy import
>> frameworks for Python have done hacks with `__getattribute__` overrides.
>>
>
> The value is more than ease of implementation. Having syntax for import
> expressions makes them statically analysable, which is needed for type
> checkers and IDE autocompletion.
>

This has been brought up a few times and I don't get it. Currently a use of
an imported module is perfectly analyzable by all the static type checkers
I know of (e.g. mypy, Pyre, pyright). For the static analyzer it makes no
difference if I have

import re
.
.
.
def foo(x):
if re.match(r"blah", x): ...

or the hypothetical inline form:

def foo(x):
if @re.match(r"blah", x): ...


> But I also see value in being able to type out code that uses modules not
> yet imported without breaking my flow to add an import statement. I don't
> yet trust IDEs to do this because I've been bitten by them doing so
> incorrectly in the past.
>

I have too.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YCZ5QZM3LUFWGAEBPIZ3LYOP4UAYJRR4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Daniel Pope]

On Fri, 8 Apr 2022, 17:44 Guido van Rossum,  wrote:

> The interesting idea here seems to make "lazy imports" easier to implement
> by making them explicit in the code. So far, most lazy import frameworks
> for Python have done hacks with `__getattribute__` overrides.
>

The value is more than ease of implementation. Having syntax for import
expressions makes them statically analysable, which is needed for type
checkers and IDE autocompletion.

But I also see value in being able to type out code that uses modules not
yet imported without breaking my flow to add an import statement. I don't
yet trust IDEs to do this because I've been bitten by them doing so
incorrectly in the past.

The key questions to me are
> - What should the notation be?
>

I would like to bid again for (import package.module) as an expression.
Instead of doing the import and assigning package to a variable package it
would evaluate to the module object package.module.

The `as` form is not needed because no name is assigned and the `from` form
isn't as valuable because you can just use attribute access afterwards.

It isn't terse but it does make use of the import keyword and is thus
instantly recognisable. It is even syntax highlighted correctly by much
existing software. If we're using the import keyword then I think it has to
look like this.

But I concede that it isn't particularly elegant to type hint things with

(import collections.abc).Mapping

...but not so inelegant that I couldn't see myself using it for a few
one-off imports per module.

A quirk is that it means there's a big difference between the statements

import foo

and

(import foo)

because one assigns a variable. I don't mind that; I don't think it is too
ambiguous to a reader.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/T6V7WZAFEXGWMHXLHS7XHYXI5OPMOZKA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Dan Stromberg]

On Fri, Apr 8, 2022 at 1:26 AM Malthe  wrote:

> This is an idea which has been brought up before, sometimes introduced
> as "heresy". But an interesting twist has surfaced now which is
> typing.
>

What for?  To save a few keystrokes?

Can't some IDE's add the import for you?

Please don't drag Python in the direction of line noise.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IOJ5J5OY4H2OLMXOGIX24EHGVT2VY5N4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Carl Meyer via Python-Dev]

Hi Malthe,

On Fri, Apr 8, 2022 at 12:04 PM Malthe  wrote:
> Actually, to me the interesting idea is not so much lazy imports – I
> think they should not be lazy, at least that was my initial thought. I
> think they should be immediately resolved before anything else in that
> module:

I'm +0.25 on your idea as simply a streamlined syntax for inline
imports (given actually finding an appropriate syntax, which I haven't
thought much about; @ probably doesn't work due to the conflict with
decorator syntax, but there might be other options.). If it existed I
would probably use it occasionally, but I don't feel a strong need for
it.

But I think your proposal is much stronger if you eliminate the
hoisting from it; with the hoisting I'd be -1. Out-of-source-order
execution like this is just quite surprising in the context of Python.

> 1. This would settle any discussion about performance impact (there
> wouldn't be any).

If the inline import is actually a performance problem because a
certain code path is very hot, the solution is simple: don't use the
inline import there, use a top-of-module import instead.

> 2. This would enable IDEs, typers and other tooling to know the type
> using existing import logic.

I don't think it enables any such thing. Static-analysis tooling has
only the source code to work with, runtime behavior doesn't affect it.
If the runtime executes these imports out-of-order, that won't make
the slightest difference to how easily IDEs and type checkers can
analyze the source code.

> 3. Catch errors early!

The very strong precedent in Python is that errors in code are caught
when the code runs, and the code runs more or less when you'd expect
it to, in source order. If you want to catch errors earlier, use a
static analysis tool to help catch them.

Carl
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ARI44O62CRMAF2IKPHJVLU5D2ADR2DP6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Carl Meyer via Python-Dev]

Hi Barry,

On Fri, Apr 8, 2022 at 12:44 PM Barry Warsaw  wrote:
>
> Start up overhead due to imports is a real problem for some class of 
> applications, e.g. CLIs, and I’ve seen a lot of hacks implemented to get 
> Python CLIs to be more responsive.  E.g. getting from invocation to —help 
> output is a major UX problem.

Definitely, we have this same problem, and also the same symptom of
people pushing hard to rewrite Python CLIs in Go for this reason.

> It’s often more complicated than just imports alone though.  Expensive module 
> scope initializations and decorators contribute to this problem.

One of our projects that can prevent much of this expensive work being
done at import time is Strict Modules[1]. Currently it's only
available as part of Cinder, though we're hoping to make it
pip-installable as part of our project to make Cinder's features more
easily accessible.

Our experience in practice, though, has been that universally lazy
imports is somewhat easier to adopt than Strict Modules, and has had a
much bigger overall impact on reducing startup time for big CLIs (and
a big web server too; as you note it's not as serious an issue for a
web server in production, but restart time still does make a
difference to dev speed / experience.) Removing slow stuff happening
at import time helps, but it'll never match the speed of not doing the
import at all! We've seen startup time improvements up to 70% in
real-world CLIs just by making imports lazy. We've also opened an
issue to discuss the possibility of upstreaming this. [2]

[1] https://github.com/facebookincubator/cinder/#strict-modules
[2] https://bugs.python.org/issue46963
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/62OTFJMAMQ2WHZ4H3TUEJTECMPJDQ557/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Barry Warsaw]

Start up overhead due to imports is a real problem for some class of 
applications, e.g. CLIs, and I’ve seen a lot of hacks implemented to get Python 
CLIs to be more responsive.  E.g. getting from invocation to —help output is a 
major UX problem.

It’s often more complicated than just imports alone though.  Expensive module 
scope initializations and decorators contribute to this problem.  Python start 
up time is one of the main drivers for rewriting CLIs in Go and other languages 
where I work.  Note that this is much less of a problem for things like web 
services or other long running applications because that start up time is 
either amortized over the lifetime of the application, or aren’t directly 
visible to the end user.

Lazy imports *might* help with this and seems aligned with the common trick of 
moving imports into functions rather than at module scope.  Faster CPython 
might help too.  But these all feel like they aren’t tackling the start up 
problem head on[1].  Lots of ideas have been discussed over the years (I 
remember some in-depth ones at the Microsoft core sprint a few years ago), and 
I’m sure there are all kinds of other tricks that people use.

However, if start up time isn’t a direct benefit of on-demand imports (a.k.a. 
declarative imports), I’m not sure how actually useful or used they will be.  I 
dunno, top-of-module imports never really bothered me that much.

-Barry

[1] I could be wrong about Faster CPython; ISTR there are some tickets on that 
project’s tracker that talk about start up times.

> On Apr 8, 2022, at 09:40, Guido van Rossum  wrote:
> 
> The interesting idea here seems to make "lazy imports" easier to implement by 
> making them explicit in the code. So far, most lazy import frameworks for 
> Python have done hacks with `__getattribute__` overrides. IIRC the Cinder 
> version even modifies the bytecode and/or the interpreter. Disregarding the 
> specific notation proposed, *if* people would be willing to mark the points 
> where they expect lazy imports explicitly, that would make implementation 
> much simpler.
> 
> The argument that "imports on top" makes code more readable seems pretty weak 
> to me. The current hacks to speed up startup already violate this rule 
> (imports inside functions), and in most cases I start reading or writing code 
> in the middle of a file (having gotten there via a search in my editor) and 
> the meaning of an import is either obvious (e.g. re.match(...)) or requires 
> another annoying search to find the definition of a certain unknown variable. 
> Tools can easily show all imports a module does.
> 
> The key questions to me are
> - What should the notation be?
> - Will users be willing to use it?
> 
> --Guido
> 
> 
> 
> On Fri, Apr 8, 2022 at 1:26 AM Malthe  wrote:
> This is an idea which has been brought up before, sometimes introduced
> as "heresy". But an interesting twist has surfaced now which is
> typing.
> 
> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports, usable anywhere using a simple
> syntax, @.
> 
> For example, `some_regex = @re.compile(...)`.
> 
> What happens then is that before anything else in that module, that
> symbol is imported:
> 
> from re import compile as _mangled_re_compile
> 
> It must be the very first thing (hoisting) because when else would it
> happen? It's been suggested before to have a shorthand syntax which
> does a dynamic import at the time of using it but this brings me to
> the twist:
> 
> We want typing to pick up these imports. And this twist has a second
> leg which is that we often need to import symbols simply in order to
> type some argument type or return type. This leads to a great many
> more imports to type.
> 
> (Nevermind that if you want to take typing further, abstract
> interfaces really make more sense rather than specific
> implementations, but the point is the same.)
> 
> A situation where this would come in really handy is in scripting such
> as how we use Python in Apache Airflow to let users write out simple
> workflows. A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
> 
> default_args = {
> "start_date": @datetime.datetime(...)
> }
> 
> It's a lot more ergonomic from a user perspective (well perhaps for
> some users and for some programs).
> 
> Thoughts?
> 
> Cheers
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/UX6EJHLJNNLMFPWVPF5ANYHQSHDZK7SV/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> --
> --Guido van Rossum (python.org/~guido)
> Pronouns: he/him (why is my pronoun here?)
> ___
> Python-Dev mailing 

[Python-Dev] Re: Declarative imports

[Carl Meyer via Python-Dev]

An interesting point in the lazy imports design space that I hadn't
previously considered could be:

- lazy imports are explicitly marked and usage of the imported name
within the module is transparent, but
- lazily imported names are _not_ visible in the module namespace;
they can't be accessed by other modules or re-exported; they are
internal-use-only within the module

This compromise would, I think, make it possible to implement lazy
imports entirely in the compiler (effectively as syntax sugar for an
inline import at every usage site), which is definitely an
implementation improvement.

I think in practice explicitly marking lazy imports would make it
somewhat harder to gain the benefits of lazy imports for e.g. speeding
up startup time in a large CLI, compared to an implicit/automatic
approach. But still could be usable to get significant benefits.

Carl
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZT6CXQPFCWZD2M65YXCSAPPGNDGA6WNE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Carl Meyer via Python-Dev]

You only get the ease-of-implementation benefit if you are willing to
explicitly mark every _use_ of a lazy-imported name as special (and
give the fully qualified name at every usage site). This is rather
more cumbersome (assuming multiple uses in a module) than just
explicitly marking an import as lazy in one location and then using
the imported name in multiple places normally.

Other "lazy import" solutions are trying to solve a problem where you
want the name to be usable (without special syntax or marking) in many
different places in a module, and visible in the module namespace
always -- but not actually imported until someone accesses/uses it.
The difficulty arises because in this case you need some kind of
placeholder for the "deferred import", but you need to avoid this
"deferred object" escaping and becoming visible to Python code without
being resolved first. Explicitly marking which imports are lazy is
fine if you want it (it's just a matter of syntax), but it doesn't do
anything to solve the problem of allowing usage of the lazy-imported
name to be transparent.

I agree that the idea that top-of-module imports help readability is
overstated; it sounds slightly Stockholm-syndrome-ish to me :)
Top-of-module imports are frankly a pain to maintain and a pain to
read (because they are often distant from the uses of the names). But
they are a necessary evil if you want a) namespaces and b) not
constantly retyping fully-qualified names at every usage site. Python
is pretty committed to namespaces at this point (and I wouldn't want
to change that), so that leaves the choice between top-of-module
imports vs fully qualifying every use of every name; pick your poison.
(Inline imports in a scope with multiple uses are a middle ground.)

Carl
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/N4T2YMPHBLJXKCFA5CIPBFIZJJKO7SHR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Malthe]

On Fri, 8 Apr 2022 at 16:40, Guido van Rossum  wrote:
> The interesting idea here seems to make "lazy imports" easier to implement by 
> making them explicit in the code. So far, most lazy import frameworks for 
> Python have done hacks with `__getattribute__` overrides. IIRC the Cinder 
> version even modifies the bytecode and/or the interpreter. Disregarding the 
> specific notation proposed, *if* people would be willing to mark the points 
> where they expect lazy imports explicitly, that would make implementation 
> much simpler.

Actually, to me the interesting idea is not so much lazy imports – I
think they should not be lazy, at least that was my initial thought. I
think they should be immediately resolved before anything else in that
module:

1. This would settle any discussion about performance impact (there
wouldn't be any).
2. This would enable IDEs, typers and other tooling to know the type
using existing import logic.
3. Catch errors early!

Notation is hard of course. Would users be willing to use it? I think
so, at least in Rust, people do use it and I think just in the right
places – typically for imports that are either "canonicalized" such as
`re.compile` is in Python, or used just once.

Cheers
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NKBWGPJRKYQX4Z4WDDD74CSV4XEBGTFD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Guido van Rossum]

The interesting idea here seems to make "lazy imports" easier to implement
by making them explicit in the code. So far, most lazy import frameworks
for Python have done hacks with `__getattribute__` overrides. IIRC the
Cinder version even modifies the bytecode and/or the interpreter.
Disregarding the specific notation proposed, *if* people would be willing
to mark the points where they expect lazy imports explicitly, that would
make implementation much simpler.

The argument that "imports on top" makes code more readable seems pretty
weak to me. The current hacks to speed up startup already violate this rule
(imports inside functions), and in most cases I start reading or writing
code in the middle of a file (having gotten there via a search in my
editor) and the meaning of an import is either obvious (e.g. re.match(...))
or requires another annoying search to find the definition of a certain
unknown variable. Tools can easily show all imports a module does.

The key questions to me are
- What should the notation be?
- Will users be willing to use it?

--Guido



On Fri, Apr 8, 2022 at 1:26 AM Malthe  wrote:

> This is an idea which has been brought up before, sometimes introduced
> as "heresy". But an interesting twist has surfaced now which is
> typing.
>
> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports, usable anywhere using a simple
> syntax, @.
>
> For example, `some_regex = @re.compile(...)`.
>
> What happens then is that before anything else in that module, that
> symbol is imported:
>
> from re import compile as _mangled_re_compile
>
> It must be the very first thing (hoisting) because when else would it
> happen? It's been suggested before to have a shorthand syntax which
> does a dynamic import at the time of using it but this brings me to
> the twist:
>
> We want typing to pick up these imports. And this twist has a second
> leg which is that we often need to import symbols simply in order to
> type some argument type or return type. This leads to a great many
> more imports to type.
>
> (Nevermind that if you want to take typing further, abstract
> interfaces really make more sense rather than specific
> implementations, but the point is the same.)
>
> A situation where this would come in really handy is in scripting such
> as how we use Python in Apache Airflow to let users write out simple
> workflows. A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
>
> default_args = {
> "start_date": @datetime.datetime(...)
> }
>
> It's a lot more ergonomic from a user perspective (well perhaps for
> some users and for some programs).
>
> Thoughts?
>
> Cheers
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/UX6EJHLJNNLMFPWVPF5ANYHQSHDZK7SV/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BTWPAJRVQ24QX2Z7TIQPDYRXKJOGRPMU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Jarek Potiuk]

I do not think it's a "typing" improvement. 

I think what I like most in the proposal (which might be implemented 
differently - I am not too tied to this) is that it potentially brings huge 
optimisations in import times when currently a lot of imports are done 
unnecessarily early - slowing down startup times of tools that require parsing 
of python code.

There are few examples of those:
* Airflow DAGs - this is one of the problems we have in Airflow (and this is I 
think one of the reasons why Malthe brought it) is nicely described here 
https://airflow.apache.org/docs/apache-airflow/stable/best-practices.html#top-level-python-code
* Tools like click-complete (where autocompletion suggestions are generated 
based on the decorators of Python code)
* Static Python Typing checks (mostly when you declare Type returned/used by 
function that is only used inside the function).

In all those cases, you really do not care about some of the imported code that 
is in some kind of  "internal scope" and your python parsing does not 
**really** need to perform the imports to do the job.

* In Airflow DAGs - tasks which are defined as functions (and default arguments 
from the example from Malthe) are not executed at all when the DAGs are parsed 
by scheduler. The DAGs are parsed to get the structure of the DAG. In this case 
we could optimise the time it take to parse the DAG and generate the structure.

* In Click-complete - we only really care about names of the parameters for 
functions, but static typing for the function parmeters or returned value are 
not needed at all to generate complete suggestions. In this case importing time 
is essential, because you want to provide autocomplete suggestions **really 
quickly**

* In case of typing of parameters or return values, it leads quite often to 
circular imports generated. If - in your module - the only place you use a type 
is inside one or two functions, adding static typing involves adding top-level 
import for that type. And that has huge potential of generating circular 
imports. Before static typing in parameters or return values of those, local 
imports would do the job nicely,  but with typing added to parameters and 
return values, you cannot do local imports in those functions that have those 
parameters or return values. This often leads to adding local imports 
elsewhere, but in fact a "logical" thing is that those imports should be local 
for the functions and any of the scopes that the functions are called. 
Top-level imports in this case are causing unnecessary cross-module 
dependencies and circular imports.

And some of those problems can currently be solved with local imports - but not 
all (default_values case by Malthe cannot).

I think this idea has really good "need" behind. Maybe the way to implement 
could be improved on, but I see clearly the use cases and needs the "general 
idea" aims to solve.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ALXSM6753M7AT6MT7KVQULUCFX5G4XT7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Victor Stinner]

While it's different than you proposal, some people may like this
magic "smart imports" project which lazily imports a module when it's
used :-)

Project: https://pypi.org/project/smart-imports/

Replace:
---

import math
from my_project import calc
# 100500 other imports

def my_code(argument, function=calc):
return math.log(function(argument))
---

with:
---
import smart_imports
smart_imports.all()
# no any other imports

def my_code(argument, function=calc):
return math.log(function(argument))
---

I don't like it :-) I prefer explicit imports at the top!

Victor
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KLXXAFVPY6RMGJVIFPJ5UNIXNO2YG472/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Steve Dower]

On 4/8/2022 12:51 PM, Daniel Pope wrote:

On Fri, 8 Apr 2022 at 12:23, Steve Dower  wrote:


I've read the rest of the thread so far, and agree strongly that we
can't do this at the language/runtime level.


You mean the hoisting, right?

I don't see any reason why an import expression without hoisting would
be impractical. But I'd like to hear your thoughts if you think it is.


Sure, __import__() or better yet, importlib.util.import_module() are 
perfectly good ways to do an import in an expression. Both have exactly 
the right amount of code-smell as well, and only a few surprising ways 
to interfere with the behaviour of apparently completely unrelated code ;)


As others pointed out, implicitly defining variables that outlive the 
evaluation of an expression is incredibly messy. It was bad enough 
getting assignment expressions defined well enough to (a) not be a 
terrible footgun and (b) be possible for the average developer to 
predict what will happen when they're used (arguably this isn't true 
either, but the "average" developer probably isn't doing things that 
will hit the edge cases).


We don't want to have to figure something like that out again. Certainly 
not for the sake of typing - the SC has already ruled that language 
development won't be driven by the convenience of static type users.


Cheers,
Steve

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SRAIPD7DQH36DNBZVEAA6I56IZHLGA66/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Paul Moore]

On Fri, 8 Apr 2022 at 12:57, Daniel Pope  wrote:
>
> On Fri, 8 Apr 2022 at 12:23, Steve Dower  wrote:
> >
> > I've read the rest of the thread so far, and agree strongly that we
> > can't do this at the language/runtime level.
>
> You mean the hoisting, right?
>
> I don't see any reason why an import expression without hoisting would
> be impractical. But I'd like to hear your thoughts if you think it is.
>
> Desirability on the other hand is subjective. I think I actually do
> desire it, others are not bothered. I don't see strong arguments as to
> why you definitely wouldn't want it in your language.

OK, I'll be explicit. I don't want this in Python. Having imports at
the top of a file makes it much easier to see your dependencies. The
current language features, combined with community wide style guides
that discourage the use of local imports, make this straightforward,
while still providing means of doing local imports if needed.

This proposal isn't just about having a new syntax to do an on-demand
import. It's also about normalising the idea that people can pull code
from other modules without declaring the intent to do so up front.
While I don't dispute that in some circumstances (notably the REPL,
and throwaway scripts[1]) not having to add import statements would be
nice, I don't see how we'd limit usage to cases like that - and I
*really* don't want to have to work out where some weird inline import
happened while debugging a priority 1 bug in a 10,000 line code base
at 2am...

Also, it should be possible to do something like this using the
existing import machinery:

from magic_import import on_demand_loader as OD
...
# Many lines of code later
default_args = {
"start_date": OD.datetime.datetime(...)
}


Trivial proof of concept implementation:

class OD:
def __getattr__(self, name):
return __import__(name)

OD = OD()

print(OD.datetime.datetime.now())

Paul

[1] Of course, today's throwaway script usually turns out to be next
month's key component in a mission-critical software stack :-(
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HC4UJPBWJHU5JWIPQQKNJRMN2QBXHJVC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Daniel Pope]

On Fri, 8 Apr 2022 at 12:23, Steve Dower  wrote:
>
> I've read the rest of the thread so far, and agree strongly that we
> can't do this at the language/runtime level.

You mean the hoisting, right?

I don't see any reason why an import expression without hoisting would
be impractical. But I'd like to hear your thoughts if you think it is.

Desirability on the other hand is subjective. I think I actually do
desire it, others are not bothered. I don't see strong arguments as to
why you definitely wouldn't want it in your language.

Design is hard, but designing this is definitely not as hard as
designing match/case or except* statements.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/G4TOKDQPJPKNEGPUGYMJFOHE7FSID2OY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Daniel Pope]

On Fri, 8 Apr 2022 at 11:26, Malthe  wrote:
> Perhaps `some_regex = re::compile(r"...")` could work.
>
> That is, :: to delineate the import.

As I mentioned in the Discourse thread, this syntax chimes with Rust
(and C++, and also PHP, triggering fond memories of Paamayim
Nekudotayim).

There are two problems, sort of on the minor side:

First, there's one place in Python where this can occur and that is in
slices: xs[start::step] means the slice from start to the end with a
step of step. But xs[(module::name)] is currently a syntax error, so
you could just require parentheses to disambiguate the case. Still, it
is confusing if you write something slightly more complex like xs[1 +
math::factorial(n)] and you get NameError: name 'math' is not defined.

Secondly, it precludes importing just a module object, you have to
import a name from a module. Maybe using a module object in an
expression is niche and can be disregarded. Or a solution could be to
allow an expression like (re::) to import a module object, which I
guess we could get used to. Also fun would be to consider what ::name
does - access to current globals without the global keyword?

I think this discussion is better suited to the python-ideas mailing
list, or the Discourse thread I linked earlier.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MO524TQHUR7PMUWOZ4PBQAXDMOOF2KFP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Steve Dower]

I've read the rest of the thread so far, and agree strongly that we 
can't do this at the language/runtime level.


However ...

On 4/8/2022 9:24 AM, Malthe wrote:

This is an idea which has been brought up before, sometimes introduced
as "heresy". But an interesting twist has surfaced now which is
typing.

[SNIP]

We want typing to pick up these imports. And this twist has a second
leg which is that we often need to import symbols simply in order to
type some argument type or return type. This leads to a great many
more imports to type.


Type checkers can do whatever they like. The syntax has to remain a 
valid Python expression, but the semantics of it can include implicit 
imports if that's what you want it to do.


I'd certainly make use of it on those occasions I actually put type 
annotations in code (rather than .pyi files).


But it doesn't need the runtime to do anything differently if the module 
isn't really going to be used at that point.


Cheers,
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/57NDZA7B3Z4PLGS5JON5YXOP44KI4LVU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Paul Moore]

On Fri, 8 Apr 2022 at 10:27, Malthe  wrote:
>
> On Fri, 8 Apr 2022 at 08:51, Paul Moore  wrote:
> > Are you exaggerating for effect here or would this *actually* just expand to
> >
> > from datetime import datetime
> > default_args = {
> > "start_date": datetime(...)
> > }
>
> Yes – and of course that is just a snippet, an actual complete script
> will have lots of such imports.
>
> The point is that in a scripting situation (especially one in which
> you have many small scripts) the
> top-level import requirement for simple imports like `datetime`
> becomes rather verbose.

But Python's origin is in scripting situations, and explicit imports
for scripts has always been both the normal approach, and one of
Python's *strengths* ("Explicit is better than implicit"). Arguing
that explicit imports are a bad thing in (general) scripts is arguing
against decades of history and experience. If there's a reason why
*your specific context* would benefit from an abbreviated form, you
need to present it. But arguing that explicit imports are too verbose
for all cases of scripting isn't going to fly, frankly.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TMJ4ZI7J62C64PZXQU3WRLFA6FAKPRVB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Malthe]

On Fri, 8 Apr 2022 at 10:15, Daniel Pope  wrote:
> But, your proposed syntax is not usable because it is ambiguous. Exactly what 
> you propose is already used for decorators in a way that the parser would not 
> be able to distinguish a decorator from an import expression. Consider:
>
> @ham.spam()   # import expression or decorator?
>
>  def eggs():
> ...
>
> This currently parses as a decorator and for backwards compatibility that 
> must not change, which means that import expressions would be usable in some 
> contexts and not others purely based on what follows them.

Perhaps `some_regex = re::compile(r"...")` could work.

That is, :: to delineate the import.

@breakfast.ham::spam
def eggs(): pass

Cheers
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VXOSHFQDLPI5KXVNXWKFMRZOZHYJLZL5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Daniel Pope]

On Fri, 8 Apr 2022, 09:30 Malthe,  wrote:

> For example, `some_regex = @re.compile(...)`.
>

I like the idea of import expressions. I pitched it on Discourse recently:
https://discuss.python.org/t/import-expressions/11582

However, I do not see hoisting as something that should be done by Python.
In that thread it was suggested your IDE could do it, or maybe isort, which
seems fine because it's at the programmer's discretion and ends up being
explicit.

But, your proposed syntax is not usable because it is ambiguous. Exactly
what you propose is already used for decorators in a way that the parser
would not be able to distinguish a decorator from an import expression.
Consider:

@ham.spam()   # import expression or decorator?

 def eggs():
...

This currently parses as a decorator and for backwards compatibility that
must not change, which means that import expressions would be usable in
some contexts and not others purely based on what follows them.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IS7WHO3ZL33D4PSHWFSLU4D4WQ5736MY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Steven D'Aprano]

On Fri, Apr 08, 2022 at 08:24:40AM +, Malthe wrote:

> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports, 

I'm not sure I understand why you consider this "declarative".

https://en.wikipedia.org/wiki/Declarative_programming

As I see it, this proposal merely adds a unary operator for implicit 
imports.



> For example, `some_regex = @re.compile(...)`.
> 
> What happens then is that before anything else in that module, that
> symbol is imported:
> 
> from re import compile as _mangled_re_compile
> 
> It must be the very first thing (hoisting) because when else would it
> happen?

On-demand. As you say yourself, this could be a dynamic import at the 
time of use.


> It's been suggested before to have a shorthand syntax which
> does a dynamic import at the time of using it but this brings me to
> the twist:
> 
> We want typing to pick up these imports.

At the moment static type checkers have to look for two import 
statements: `import spam`, and `from spam import eggs`.


With this proposal, they will still need to look for those two import 
statements, but they will also need to parse every expression looking 
for `@` as a unary operator:

result = 2*x**3 - 3*y + @math.sin(x*y) - 5*x*y**2

This does not help typing at all. It just means the type-checker has to 
do more work to recognise imports.



> And this twist has a second
> leg which is that we often need to import symbols simply in order to
> type some argument type or return type. This leads to a great many
> more imports to type.

Code is read much more than it is written. I would much prefer to have 
explicit imports (by convention, if not necessity) in one place at the 
top of the page, than to mystery symbols where the implicit import could 
be buried far away.

Right now, this is an error:

# Start of module.
obj = wibble.foo

because wibble is not a built-in nor a global (excluding weird 
shenanigans committed by other modules), so the name "wibble" doesn't 
exist. But with hoisting, that import could be *anywhere* in the file. 
Even in dead code.

# Start of module.
obj = wibble.foo
...
...
# five pages of code later
for a in obj.method():
while flag:
if condition:
@wibble

That's right, with hoisting you can use a module before you import it. 
Mind. Blown.

Have pity on beginners, casual Python programmers, and the people who 
have to help them. Don't implement this horror.

If it were merely on-demand imports, then we would know that the import 
@wibble would have to appear in actual, executed code before you can use 
wibble. But with hoisting, we don't even have that promise.

It is truly spooky action at a distance. And to save nothing but an 
import line.


> A situation where this would come in really handy is in scripting such
> as how we use Python in Apache Airflow to let users write out simple
> workflows. A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
> 
> default_args = {
> "start_date": @datetime.datetime(...)
> }

That's just a three-liner, which becomes a four-liner:

import datetime
default_args = {
"start_date": datetime.datetime(...)
}



-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GILL25XYSAPF4FN7LTZC7XLDB7ZX4E4Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Malthe]

On Fri, 8 Apr 2022 at 08:51, Paul Moore  wrote:
> Are you exaggerating for effect here or would this *actually* just expand to
>
> from datetime import datetime
> default_args = {
> "start_date": datetime(...)
> }

Yes – and of course that is just a snippet, an actual complete script
will have lots of such imports.

The point is that in a scripting situation (especially one in which
you have many small scripts) the
top-level import requirement for simple imports like `datetime`
becomes rather verbose.

Thanks
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FVYRPSN6A7KN73MTQIF2UIU5W3HSUPGV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Paul Moore]

On Fri, 8 Apr 2022 at 09:29, Malthe  wrote:

> A workflow definition which could be a 5-liner quickly
> becomes a 20-liner – consider for example:
>
> default_args = {
> "start_date": @datetime.datetime(...)
> }

Are you exaggerating for effect here or would this *actually* just expand to

from datetime import datetime
default_args = {
"start_date": datetime(...)
}

I'm not suggesting that there isn't a genuine benefit here, but your
example doesn't demonstrate what it is.

Paul
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RRLFDL2BDF3UV646GVZVRXUSGM7X5FRQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Declarative imports

[Chris Angelico]

On Fri, 8 Apr 2022 at 18:29, Malthe  wrote:
>
> This is an idea which has been brought up before, sometimes introduced
> as "heresy". But an interesting twist has surfaced now which is
> typing.
>
> But firstly, let me present the idea. It is very simple, that Python
> should have declarative imports, usable anywhere using a simple
> syntax, @.
>
> For example, `some_regex = @re.compile(...)`.
>
> What happens then is that before anything else in that module, that
> symbol is imported:
>
> from re import compile as _mangled_re_compile
>
> It must be the very first thing (hoisting) because when else would it
> happen?

JavaScript has this feature. ES6 modules are always imported
declaratively and always before the rest of the script is executed.

It is *extremely* annoying. Most of the time, it behaves the same way
as Python's dynamic imports, but any time it doesn't, the "imports
before the rest of the code" feature is nothing but hassle.

Here's an alternative: Have a whitelist of modules that you make use
of in this way. Any time you refer to a name that's on that whitelist
and doesn't have an assigned meaning in the module, insert an import
statement at the top of the script. It's not too hard to set up your
editor to be able to do this (eg in SciTE, I can press Ctrl+1 to
quickly run a script; a lot of editors let you configure a code
formatter that runs automatically on save, which would also work), and
then you retain full control of exactly where the importing happens.

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YUNFSHIONZNLZ6QS3GWF4QVLP4IWZY6W/
Code of Conduct: http://python.org/psf/codeofconduct/