[Python-ideas] Re: Add an export keyword to better manage __all__

[Joseph Martinot-Lagarde]

Theia Vogel wrote:
> Hi,
> I was refactoring some code today and ran into an issue that always bugs me 
> with
> Python modules. It bugged me enough this time that I spent an hour banging 
> out this
> potential proposal to add a new contextual keyword. Let me know what you 
> think!
> Theia

I like the @public decorator like 
https://public.readthedocs.io/en/latest/index.html (which came from 
https://bugs.python.org/issue22247). The fact that it's at the definition of a 
function (or constant) makes it quite enjoyable to use.

Joseph
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CSVJBWKLWH2W5RXCVLDK2XVYDAXMW6EZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add an export keyword to better manage __all__

[Jonathan Crall]

I'd like to give a shoutout to a package I wrote called mkinit, which helps
autogenerate explicit `__init__.py` files to mitigate some of the issues
you mentioned (especially with respect to `from blah import *`).

https://github.com/Erotemic/mkinit

On Sat, Apr 10, 2021 at 3:08 PM Guido van Rossum  wrote:

> On Sun, Mar 14, 2021 at 10:55 PM Ethan Furman  wrote:
>
>> I completely agree with this.  One of the hallmarks of Python is the
>> ability to query, introspect, and modify Python code.  It helps with
>> debugging, with experimenting, with fixing.  Indeed, one of the few
>> frustrating bits about Python is the inability to work with the C portions
>> as easily as the Python portions (note: I am /not/ suggesting we do away
>> with the C portions).
>>
>> What would be the benefits of locking down modules in this way?
>
>
> I owe you an answer here. For large projects with long lifetimes that
> expect their API to evolve, experience has taught  that any part of the API
> that *can* be used *will* be used, even if it was clearly not intended to
> be used. And once enough users have code that depends on reaching into some
> private part of a package and using a method or attribute that was
> available even though it was undocumented or even clearly marked as
> "internal" or "private", if an evolution of the package wants to remove
> that method or attribute (or rename it or change its type, signature or
> semantics/meaning), those users will complain that the package "broke their
> code" by making a "backwards incompatible change." For a casually
> maintained package that may not be a big deal, but for a package with
> serious maintainers this can prevent ordered evolution of the API,
> especially since a package's developers may not always be aware of how
> their internal attributes/methods are being used or perceived by users. So
> experienced package developers who are planning for the long term are
> always looking for ways to prevent such situations (because it is
> frustrating for both users and maintainers). Being able to lock down the
> exported symbols is just one avenue to prevent disappointment in the future.
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/XDLAAQKWDSPOJ6HKVTBEZEK6KYM5PQGP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
-Dr. Jon Crall (him)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TNTM3XCN6VBL3WT7ZT5VFG2QJ6FNJ2LM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add an export keyword to better manage __all__

[Guido van Rossum]

On Sun, Mar 14, 2021 at 10:55 PM Ethan Furman  wrote:

> I completely agree with this.  One of the hallmarks of Python is the
> ability to query, introspect, and modify Python code.  It helps with
> debugging, with experimenting, with fixing.  Indeed, one of the few
> frustrating bits about Python is the inability to work with the C portions
> as easily as the Python portions (note: I am /not/ suggesting we do away
> with the C portions).
>
> What would be the benefits of locking down modules in this way?


I owe you an answer here. For large projects with long lifetimes that
expect their API to evolve, experience has taught  that any part of the API
that *can* be used *will* be used, even if it was clearly not intended to
be used. And once enough users have code that depends on reaching into some
private part of a package and using a method or attribute that was
available even though it was undocumented or even clearly marked as
"internal" or "private", if an evolution of the package wants to remove
that method or attribute (or rename it or change its type, signature or
semantics/meaning), those users will complain that the package "broke their
code" by making a "backwards incompatible change." For a casually
maintained package that may not be a big deal, but for a package with
serious maintainers this can prevent ordered evolution of the API,
especially since a package's developers may not always be aware of how
their internal attributes/methods are being used or perceived by users. So
experienced package developers who are planning for the long term are
always looking for ways to prevent such situations (because it is
frustrating for both users and maintainers). Being able to lock down the
exported symbols is just one avenue to prevent disappointment in the future.

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XDLAAQKWDSPOJ6HKVTBEZEK6KYM5PQGP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add an export keyword to better manage __all__

[Neil Girdhar]

I like the idea of improving the way interfaces are exported in Python.  I 
still don't know what the standard is today.

Some of my favourite projects like Jax have started tucking their source in 
a _src directory: https://github.com/google/jax/tree/master/jax/_src, and 
then importing the exported interface in their main 
project: https://github.com/google/jax/blob/master/jax/scipy/signal.py.  As 
far as I know, this "private source" method is nicest way to control the 
interface.

What I've been doing is importing * from my __init__.py 
(https://github.com/NeilGirdhar/tjax/blob/master/tjax/__init__.py) and then 
making sure every module has an __all__.  This "init-all" method is 
problematic if you want to export a symbol with the same name as a folder.  
It also means that all of your modules and folders are exported names. If I 
started again, I'd probably go with the way the Jax team did it.

I agree that the proposed export keyword is nicer than __all__, and 
accomplishes what I've been doing more elegantly.  However, it suffers from 
the same extraneous symbol problem.  I would still choose the private 
source method over export.

I'm curious if anyone has ideas about how exporting an interface should be 
done today, or could be done tomorrow.
On Monday, March 15, 2021 at 9:48:11 AM UTC-4 Rob Cliffe via Python-ideas 
wrote:

>
>
> On 15/03/2021 05:54, Ethan Furman wrote:
> > On 3/14/21 12:42 PM, Stestagg wrote:
> >
> >> The value of being able to (in specific cases) reach into third-party 
> >> code, and customize it to work for your specific situation should not 
> >> be disregarded.
> >
> > I completely agree with this.  One of the hallmarks of Python is the 
> > ability to query, introspect, and modify Python code.  It helps with 
> > debugging, with experimenting, with fixing.  Indeed, one of the few 
> > frustrating bits about Python is the inability to work with the C 
> > portions as easily as the Python portions (note: I am /not/ suggesting 
> > we do away with the C portions).
> An emphatic +1.
> Rob Cliffe
> >
> >
> > What would be the benefits of locking down modules in this way?
> >
> > -- 
> > ~Ethan~
> > ___
> > Python-ideas mailing list -- python...@python.org
> > To unsubscribe send an email to python-id...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at 
> > 
> https://mail.python.org/archives/list/python...@python.org/message/KIFAVTRMIPCVAX2RZ7NDSTDM46AHSDWK/
>  
> 
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-ideas mailing list -- python...@python.org
> To unsubscribe send an email to python-id...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python...@python.org/message/TFGG5LBGEIH34WJNIKZGBCT67A6KFGKM/
>  
> 
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YDTNI7V2V6L3SEJCHOYZWSXC3QVB4WBS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python Idea - extension of 'with'

[Stephen J. Turnbull]

Serhiy Storchaka writes:

 > A year or two ago I proposed the same syntax with different semantic: to
 > catch only exceptions in the context manager, not in the with
 > block.

FWIW, this is the semantics I would expect, for the reason you give:

 > Exceptions in the with block you can catch by adding try/except
 > around the with block, exceptions in the with block and the context
 > manager you can catch by adding try/except around the with
 > statement, but there is no currently way to catch only exceptions
 > in the context manager.

Does this apply to the necessary setup in any other syntactic constructs?
For example, in the abstract this is a "problem" in 'for':

>>> for i in [1]:
... for j in 2:
... pass
...
Traceback (most recent call last):
  File "", line 2, in 
TypeError: 'int' object is not iterable

it's not possible to catch the TypeError *only* in the outer loop.  Of
course this isn't a "real" problem in 'for', since this TypeError is
surely not an exceptional situation you need to catch and handle at
runtime, it's just a bug.

I guess the biggest difference is that in 'with' the setup
(corresponding to calling 'iter' on the iterable in 'for') is
specified to be arbitrary code and can be arbitrarily complex.  'with'
is the only construct where that's true, that I know of.  (Of course
the setups for other syntactic constructs are very simple.)  Is that
so?

Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WUT5FZDXRGCZB6TVJYAO2NKHH4A33HIJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding syntax for the empty set

[Stephen J. Turnbull]

Chris Angelico writes:

 > Not to mention everyone's keyboards. Python != APL. Err, I mean,
 > Python ≠ APL.

I am really tired of this argument.

set() is not going to go away.  And it's easy enough to install
input methods (aka "keyboards") on any modern system that allow you to
enter these if you want to.  Most programmer's editors allow you to
define abbreviations that will do the same.  It's just not that hard
if you want to do it, and there's no way forcing you to do it will
make it past the backward-compatibility requirement.

I'm perfectly happy with the TOOWTDI/YAGNI argument: you don't buy
much clarity with ∅ over set(), with math.π over math.pi, or λx: x
over lambda x: x, so let's just don't.[1]

But let's not introduce the purely English native problem of
undeveloped muscle memory when we're discussing whether Python should
allow these aliases (and aliases they have to be for backward
compatibility).  On the flip side of the TOOWTDI argument, I'm pretty
sure most programmers will have no trouble understanding any of the
aliases mentioned above.  Sickly sweet syntactic sugar, yes, but not a
barrier to understanding.

Since many of the usual identifiers actually work already:

>>> import math
>>> π = math.pi; π
3.141592653589793
>>> τ = math.tau; τ
6.283185307179586
>>> γ = math.gamma; γ(1)
1.0

we are going to start seeing them.  I'm -1 on putting these in the
stdlib; programmers who want to do this can do it already.  I don't
think that fact that

>>> γ


uses the English spelling rather than the Greek letter is a big deal,
and isn't justification for adding them to the stdlib.

To me, the question here is if we should enable this compact,
traditional mathematical style for those who want it in their own code
by

(a) special-casing some characters that are "mis-classified" by
Unicode, such as ∞ and ∅, or perhaps
(b) add a function that allows users to override the Unicode standard
classification and add these characters to the set allowed in
identifiers.

lambda is a different issue.  We'd have to special-case the Greek
letter version and probably add it to the grammar, so that's pretty
clearly out on TOOWTDI/YAGNI grounds IMO.

Note that (b) might induce a security issue since depending on
implementation it could allow overriding compatible and confusable
normalization.  I don't think we do such normalization, but if we do,
that would be something to think about.

Steve


Footnotes: 
[1]  It's amusing that lambda is a binding *operator* in Python (thus
the lack of a space in "λx" above), but "λ" is not an operator in
Unicode -- the opposite of the issue with "∅", except that you can't
add operator symbols in Python so aliasing by the user can't work anyhow.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VSSB4HTCH3O6IRGKBDRI7FGIHN4O6OUT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding syntax for the empty set

[Guido van Rossum]

Please. Use. set().

On Sat, Apr 10, 2021 at 02:03 Serhiy Storchaka  wrote:

> 09.04.21 19:08, micro codery пише:
> >
> > You can now use `{*()}` as a syntax for empty set.
> >
> > I saw that in the ast module and think it's clever, mainly in a good
> > way. I don't think it is the same as having dedicated syntax for the
> > empty set partly because I think it needs to be taught. I don't think a
> > new pythonista would turn to empty tuple unpacking to get the empty set,
> > where I do think that either set() or {,} would be natural, at least
> > after some trial and exceptions.
>
> Do you think that {,} does not need to be taught? It is a new special
> syntax which needs paragraphs in tutorial and language reference. In
> contrary, {*()} is a simple combination of already described syntax
> elements.
>
> > It also doesn't give quite the
> > optimization as {,}.
>
> It is a trivial optimization. It was not implemented yet only because
> such code is never used in tight loops, empty set creation is very rare
> operation at all. We prefer to keep the compiler simpler and focus on
> optimizing common operations which has significant effect.
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ASBHWYDCPR3D2FCVLGPHELPRJUJOWHLL/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MWBML4BCZ5I3OR2L64JW7WZLZSUTU6GJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Non-blocking concurrent.futures ThreadPoolExecutor

[Brunoais]

(I know this has some time but I think it makes sense in context)

That is much easier said than done in real life when the whole project
is doing by a team and you are stuck with whatever you all do in the group.

Even then, I prefer making my own programs with a mix of daemon and
non-daemon threads to do the word.
Daemon threads do work that can be interrupted at any time for any reason.
Non-daemon threads are more well behaved and focus on doing their short
job with hard IO.

In my case, the ThreadPoolExecutor does most of the work for me for
these cases. Unfortunately, I think I'll have to implement my own for
daemon threads.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RJWUWBDHDLNF6Y67YXYK7IEKJJN5BAF4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Python Idea - extension of 'with'

[Serhiy Storchaka]

09.04.21 21:02, Guido van Rossum пише:
> But if there are two proposals with conflicting semantics for the same
> syntax that kills both ideas, doesn’t it? Because apparently it’s not
> clear what the syntax should mean.

Maybe. But there was also different idea for the "with" statement (with
semantic similar to Pascal's).

I understand the drawback of this syntax (error handlers are located too
far from the guarded code), but I do not have other syntax which would
look Pythonic.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DGPAYJWL3CUYNGTZ65S6HA7ADW5OFRFZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding syntax for the empty set

[Serhiy Storchaka]

09.04.21 19:08, micro codery пише:
> 
> You can now use `{*()}` as a syntax for empty set.
> 
> I saw that in the ast module and think it's clever, mainly in a good
> way. I don't think it is the same as having dedicated syntax for the
> empty set partly because I think it needs to be taught. I don't think a
> new pythonista would turn to empty tuple unpacking to get the empty set,
> where I do think that either set() or {,} would be natural, at least
> after some trial and exceptions.

Do you think that {,} does not need to be taught? It is a new special
syntax which needs paragraphs in tutorial and language reference. In
contrary, {*()} is a simple combination of already described syntax
elements.

> It also doesn't give quite the
> optimization as {,}.

It is a trivial optimization. It was not implemented yet only because
such code is never used in tight loops, empty set creation is very rare
operation at all. We prefer to keep the compiler simpler and focus on
optimizing common operations which has significant effect.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ASBHWYDCPR3D2FCVLGPHELPRJUJOWHLL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding syntax for the empty set

[Serhiy Storchaka]

09.04.21 19:42, Ricky Teachey пише:
> On Fri, Apr 9, 2021 at 3:20 AM Serhiy Storchaka
>  > wrote:
> You can now use `{*()}` as a syntax for empty set.
> 
> Interestingly, Raymond Hettinger recently had a post on twitter
> specifically deriding this usage as obfuscatory, and expressing his
> preference that people not do it (and use set() instead).

I agree, but between introducing a new syntax for special rare case and
using uncommon (but understable) expression in existing syntax I prefer
the latter.

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BL5B36MCZ3ZYU4ECYMVP5AAPU3PBTDDD/
Code of Conduct: http://python.org/psf/codeofconduct/