[Python-ideas] Re: Add an export keyword to better manage __all__

So I think there are multiple behaviors that are being described here and I think there is validity in being able to potentially patch/modify definitions when absolutely necessary, but I wonder if there's room here for something in-between (also, I find the 'import export' or 'export import'

[Python-ideas] Re: Add an export keyword to better manage __all__

On Mon, 12 Apr 2021 at 21:27, Brendan Barnwell wrote: > > On 2021-04-12 03:13, Stéfane Fermigier wrote: > > The public API of a library is the one which is documented as > > such. > > > > Right, except that in practice: > > > > 1) Many useful libraries are not documented or properly

[Python-ideas] Re: Add an export keyword to better manage __all__

Wow, super helpful response. On Mon, Apr 12, 2021 at 1:26 PM Brendan Barnwell wrote: > On 2021-04-12 03:13, Stéfane Fermigier wrote: > > The public API of a library is the one which is documented as > > such. > > > > > > Right, except that in practice: > > > > 1) Many useful libraries

[Python-ideas] Re: Add an export keyword to better manage __all__

On 2021-04-12 03:13, Stéfane Fermigier wrote: The public API of a library is the one which is documented as such. Right, except that in practice: 1) Many useful libraries are not documented or properly documented. Then they're buggy. I'm not convinced by the argument that "in

[Python-ideas] Re: Add an export keyword to better manage __all__

This is a great discussion. However, there's one issue that seems to be getting lost. Consider this library structure: test/lemon test/lemon/__init__.py test/module.py where test/module.py is: __all__ = ['lemon'] def lemon(): pass and test/__init__.py is: from .module import * from

[Python-ideas] Re: Add an export keyword to better manage __all__

M.-A. Lemburg wrote: > The public API of a library is the one which is documented as > such. That's really all there is to it. Documentation is written > explicitly by the author of a package and languages provides > a lot more nuances than using some programmatic mechanism. I use __all__ with

[Python-ideas] Re: Add an export keyword to better manage __all__

On Mon, 12 Apr 2021 at 11:41, M.-A. Lemburg wrote: > > Right, except that in practice: > > > > 1) Many useful libraries are not documented or properly documented. > > In those cases, I'd argue that such libraries then do not really care > for a specific public API either :-) > > > 2) People

[Python-ideas] Re: Add an export keyword to better manage __all__

On 12.04.2021 12:13, Stéfane Fermigier wrote: > > > On Mon, Apr 12, 2021 at 11:39 AM M.-A. Lemburg > wrote: > > I don't think that trying to do this programmatically via some > kind of language construct is a good approach. > > > It's not the only approach,

[Python-ideas] Re: Add an export keyword to better manage __all__

On Mon, Apr 12, 2021 at 11:39 AM M.-A. Lemburg wrote: > I don't think that trying to do this programmatically via some > kind of language construct is a good approach. > It's not the only approach, but I argue that it's useful. Two use cases: - IDE's that will nudge the developer towards

[Python-ideas] Re: Add an export keyword to better manage __all__

I don't think that trying to do this programmatically via some kind of language construct is a good approach. The public API of a library is the one which is documented as such. That's really all there is to it. Documentation is written explicitly by the author of a package and languages provides

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sun, Apr 11, 2021 at 2:07 PM Oscar Benjamin wrote: > On Sun, 11 Apr 2021 at 10:25, Stéfane Fermigier wrote: > > > 2) What's a "public API" ? A particular library can have an internal API > (with regular public functions and methods, etc,) but only willingly expose > a subset of this API to

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sun, 11 Apr 2021 at 10:25, Stéfane Fermigier wrote: > > On Sat, Apr 10, 2021 at 9:08 PM Guido van Rossum wrote: >> >> >> I owe you an answer here. For large projects with long lifetimes that expect >> their API to evolve, experience has taught that any part of the API that >> *can* be used

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sat, Apr 10, 2021 at 9:08 PM Guido van Rossum wrote: > > I owe you an answer here. For large projects with long lifetimes that > expect their API to evolve, experience has taught that any part of the API > that *can* be used *will* be used, even if it was clearly not intended to > be used.

[Python-ideas] Re: Add an export keyword to better manage __all__

Theia Vogel wrote: > Hi, > I was refactoring some code today and ran into an issue that always bugs me > with > Python modules. It bugged me enough this time that I spent an hour banging > out this > potential proposal to add a new contextual keyword. Let me know what you > think! > Theia I

[Python-ideas] Re: Add an export keyword to better manage __all__

I'd like to give a shoutout to a package I wrote called mkinit, which helps autogenerate explicit `__init__.py` files to mitigate some of the issues you mentioned (especially with respect to `from blah import *`). https://github.com/Erotemic/mkinit On Sat, Apr 10, 2021 at 3:08 PM Guido van

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sun, Mar 14, 2021 at 10:55 PM Ethan Furman wrote: > I completely agree with this. One of the hallmarks of Python is the > ability to query, introspect, and modify Python code. It helps with > debugging, with experimenting, with fixing. Indeed, one of the few > frustrating bits about Python

[Python-ideas] Re: Add an export keyword to better manage __all__

I like the idea of improving the way interfaces are exported in Python. I still don't know what the standard is today. Some of my favourite projects like Jax have started tucking their source in a _src directory: https://github.com/google/jax/tree/master/jax/_src, and then importing the

[Python-ideas] Re: Add an export keyword to better manage __all__

On 15/03/2021 05:54, Ethan Furman wrote: On 3/14/21 12:42 PM, Stestagg wrote: The value of being able to (in specific cases) reach into third-party code, and customize it to work for your specific situation should not be disregarded. I completely agree with this.  One of the hallmarks of

[Python-ideas] Re: Add an export keyword to better manage __all__

On 3/14/21 12:42 PM, Stestagg wrote: The value of being able to (in specific cases) reach into third-party code, and customize it to work for your specific situation should not be disregarded. I completely agree with this. One of the hallmarks of Python is the ability to query, introspect,

[Python-ideas] Re: Add an export keyword to better manage __all__

Hi Steve, I don't think I can explain adequately why I challenged you, so I'll just apologize. Your feedback (about the 'export' proposal, and about my challenge of your credentials) is duly noted. Sorry! --Guido On Sun, Mar 14, 2021 at 4:11 PM Stestagg wrote: > On Sun, Mar 14, 2021 at 8:22

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sun, Mar 14, 2021 at 8:22 PM Guido van Rossum wrote: > If you feel so strongly about it maybe we need to see some credentials. > What's your background? (This may sound like an odd request, but reputation > matters, and right now I have no idea who you are or why I should take your > opinion

[Python-ideas] Re: Add an export keyword to better manage __all__

If you feel so strongly about it maybe we need to see some credentials. What's your background? (This may sound like an odd request, but reputation matters, and right now I have no idea who you are or why I should take your opinion seriously, no matter how eloquently it is stated.) Note that C

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sun, 14 Mar 2021 at 18:58, Guido van Rossum wrote: > On Sun, Mar 14, 2021 at 7:11 AM Theia Vogel wrote: > >> > import the_module >> >> > the_module.sys >> >> > would work, but >> >> > from the_module import sys >> >> > would not work? >> >> > That might be odd and confusing. >> >> Good

[Python-ideas] Re: Add an export keyword to better manage __all__

On Sun, Mar 14, 2021 at 7:11 AM Theia Vogel wrote: > > import the_module > > > the_module.sys > > > would work, but > > > from the_module import sys > > > would not work? > > > That might be odd and confusing. > > Good point. I'm not familiar with CPython internals so I'm not sure how > this

[Python-ideas] Re: Add an export keyword to better manage __all__

> import the_module > the_module.sys > would work, but > from the_module import sys > would not work? > That might be odd and confusing. Good point. I'm not familiar with CPython internals so I'm not sure how this would work on the implementation side, but I definitely think it would be

[Python-ideas] Re: Add an export keyword to better manage __all__

> You write about auto-populating `__all__`. I am not aware of it ever auto-populating. What are you referring to here? (The behavior that in the absence of `__all__`, everything not starting with `_` is exported, is not auto-population -- it's a default behavior implemented by `import *`, not by

[Python-ideas] Re: Add an export keyword to better manage __all__

> > > I'm not sure that I would let `export` use the existing `__all__` > machinery anyway. Maybe in a module that uses `export` there should be a > different rule that disallows importing anything from it that isn't > explicitly exported, regardless of what form of import is used (`__all__` >

[Python-ideas] Re: Add an export keyword to better manage __all__

Yeah, all the shenanigans with `__all__` make it clear that it's the wrong solution, and we should do something better. Fortunately the PEG parser and its "soft keywords" feature (debuting for match/case in 3.10) makes it much easier to do this. I had thought about this and came up with similar