[Python-Dev] Re: PEP 654 except* formatting

2021-10-04 Thread Terry Reedy

On 10/4/2021 9:57 AM, Ammar Askar wrote:

Throwing in another +1 for `except group`.

It's explicit, doesn't introduce new punctuation and avoids confusion 
with unpacking.


I agree for same reasons.  And avoids more bikeshedding.

I checked and if 'except group' is added to keyword.kwlist *before* 
'except', the pair is recognized as a keyword phrase by IDLE's syntax 
highlighter without any change.  ('except\s*group' would take care of 
variable spacing)



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EYS6Q53UN2KDBH2VM4KA7DVRL76KJYVX/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Changing exception text in micro releases

2021-09-24 Thread Terry Reedy

On 9/24/2021 10:46 AM, Eric V. Smith wrote:

On 9/24/2021 10:39 AM, Ethan Furman wrote:

On 9/24/21 6:51 AM, Eric V. Smith wrote:


> This is clearly an improvement. My question is: is it okay to 
backport this to 3.9 and 3.10? I
> think we have a rule that it's okay to change the exception text, 
but I'm not sure how that

> applies to micro releases (3.9.x, 3.10.1).

"better" != "bug fix", so I wouldn't change it in a micro release.


Yeah, as much as I'd love to see this included, you're right. Thanks for 
the "adult" take on it.


Exception messages are subject to change in each new version, without 
deprecating the old version.  Changes are sometimes backported when the 
existing message is sufficiently erroneous. 'Sufficiently' means 
something like 'harm of error outweighs harm of changing'.  I don't 
think a simple, easily recognized, typo or grammar error qualifies. 
Among active developers, I believe Guido has the most continuity of 
experience with such discussions and decisions.  Perhaps something could 
usefully be added to the devguide, but I am not sure.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KKZVPESKEAIL4DYEIGSTPLKE34KH7XTQ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: f-strings in the grammar

2021-09-21 Thread Terry Reedy

On 9/21/2021 3:29 PM, Guido van Rossum wrote:
On Tue, Sep 21, 2021 at 11:49 AM Eric V. Smith > wrote:


[Pablo]

* The parser will allow nesting quote characters. This means that
we **could** allow reusing the same quote type in nested expressions
like this:

f"some text { my_dict["string1"] } more text"

I'm okay with this, with the caveat that I raised in another email:
the effect on non-Python tools and alternate Python implementations.
To restate that here: as long as we survey some (most?) of the
affected parties and they're okay with it (or at least it doesn't
cause them a gigantic amount of work), then I'm okay with it. This
will of course be subjective. My big concern is tools that today use
regex's (or similar) to recognize f-strings, and then completely
ignore what's inside them. They just want to "skip over" f-strings
in the source code, maybe because they're doing some sort of
source-to-source transpiling, and they're just going to output the
f-strings as-is. It seems to me we're creating a lot of work for
such tools. Are there a lot of such tools? I don't know: maybe there
are none.


I assume this is primarily an issue for syntax highlighters, which must 
work under adverse conditions: the code may contain syntax errors nearby 
and they must update fast when the user is typing. (I recall these were 
the challenges when I implemented the first syntax coloring for IDLE 
several decades ago.)


If same-quote nesting were limited to 1 deep, REs could handle it. 
Since nesting is not, and same-quote nesting would not be, they cannot 
in general.


f'''length is {len(3*f"{f'{a}'}")}'''
'length is 3'

Still, if this arrives, I would consider a patch to handle the first 
nesting level.


If the syntax highlighter shows the wrong colors in an edge case, users 
can usually live with that.


Since IDLE is a gift, not a product, I've decided a feature falling 
short of perfection is OK.


Something that just colors the entire 
f-string, including the interpolations, with the "string" color is not 
optimal anyways;


To me, there is a tradeoff.  Thunderbird bird displays the gmane version 
of the example below highlighted.  I find the broken chunks to jarring.


the editor I currently use, VS Code, knows about 
f-strings and colorizes (and otherwise analyzes) the interpolations as 
expressions.


The red underline on the original display is a nice touch.  It would 
definitely help to tie the whole string together. Assuming VS Code 
handles the double nesting, does it give two underlines for the example 
above, for the outer and middle strings?
I imagine if you have a simple-minded highlighter that just uses a regex 
that matches string quotes, it will take something like my example and 
color it "string" until the first nested quote, then be confused for a 
bit, and then start coloring "string" after the second

nested quote, until the end of the f-string. So the confusion is local.


This is what IDLE does now.

I created a gist with my example. This uses some well-known colorizer 
written in JavaScript (I presume). It seems to actually already support 
nested quotes?!
https://gist.github.com/gvanrossum/b8ca09175a0d1399a8999f13c7bfa616 



And here's a copy-paste from VS Code (it also shows a red underline 
under the entire f-string, but the copy doesn't show it):


def generate(source):
 print("# What comes before")
 print(f"{source.removesuffix(".py")}.c: $(srcdir)/{source}")
 print("\t$(COMMAND)")


So these two tools, at least, seem to be doing all right (maybe because 
they both come from the JavaScript culture, where nested interpolations 
are well-known).


With only 1 or even 2 types of quotes, reusing them would be more 
necessary than it is in Python.



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SUR5PQY7MBR4S6EYKLALMXFKINPZSJLS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: f-strings in the grammar

2021-09-20 Thread Terry Reedy

On 9/20/2021 11:48 AM, Eric V. Smith wrote:


When I initially wrote f-strings, it was an explicit design goal to be 
just like existing strings, but with a new prefix. That's why there are 
all of the machinations in the parser for scanning within f-strings: the 
parser had already done its duty, so there needed to be a separate stage 
to decode inside the f-strings. Since they look just like regular 
strings, most tools could add the lowest possible level of support just 
by adding 'f' to existing prefixes they support: 'r', 'b', 'u'.


Which is what I did with IDLE.  Of course 'just add' was complicated by 
uppercase being allowed and 'f' being compatible with 'r' but not 'u' or 
'b'.


I definitely share your concern about making f-strings more complicated 
to parse for tool vendors: basically all editors, alternative 
implementations, etc.: really anyone who parses python source code. But 
maybe we've already crossed this bridge with the PEG parser.


I think we are on the far side of the bridge with contextual keywords. 
I don't believe the new code for highlighting the new match statement is 
exactly correct.  As I remember, properly classifying '_' in all the 
examples we created was too difficult, and maybe not possible.


Although I 
realize there's a difference between lexing and parsing. While the PEG 
parser just makes parsing more complicated, this change would make what 
was lexing into a more sophisticated parsing problem.


I have no love for the RE code.  I would try ast.parse if I was not sure 
it would be too slow.  I would be happy if a simplified and fast minimal 
lexer/parser were added for everyone to use.  It would not have to make 
exactly the same distinctions that IDLE currently does.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SAYU6SMP4KT7G7AQ6WVQYUDOSZPKHJMS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: f-strings in the grammar

2021-09-20 Thread Terry Reedy

On 9/20/2021 8:46 AM, Serhiy Storchaka wrote:

20.09.21 14:18, Pablo Galindo Salgado пише:

* The parser will likely have "\n" characters and backslashes in
f-strings expressions, which currently is impossible:


What about characters "\x7b", "\x7d", "\x5c", etc?

What about newlines in single quotes? Currently this works:

f'''{1 +
2}'''

But this does not:

f'{1 +
2}'


The later is an error with or without the 'f' prefix and I think that 
this should continue to be the case.



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Z2IQGYH77V72D7TEDMIFWOHTN4MHKIAB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: f-strings in the grammar

2021-09-20 Thread Terry Reedy

On 9/20/2021 7:18 AM, Pablo Galindo Salgado wrote:

there are some interesting things we **could** (emphasis on could) get 
out of this and I wanted

to discuss what people think about them.

* The parser will allow nesting quote characters. This means that we 
**could** allow reusing the same quote type in nested expressions

like this:

f"some text { my_dict["string1"] } more text"


I believe that this will disable regex-based processing, such as syntax 
highlighters, as in IDLE.  I also think that it will be sometimes 
confusing to human readers.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TWSJKE4KKSW7YD3OCHKGKJC52VUG6FY5/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Should the definition of an "(async) iterator" include __iter__?

2021-09-16 Thread Terry Reedy

On 9/16/2021 3:02 AM, Paul Moore wrote:


The debate here is (I think!) whether an *iterator* that is not also
an *iterable* is a valid iterator.


This framing of the question seems biased in that it initially uses 
'iterator' to mean 'object with __next__ but not __iter__' whe the 
propriety of that equating is at least half of the debate.



IMO it is valid (because that's what the definitions say, basically)


The definitions pretty much answer the question above in the negative.

https://www.python.org/dev/peps/pep-0234/
C-API:
 "Iterators ought to implement the tp_iter slot as returning a 
reference to themselves; this is needed to make it possible to use an 
iterator (as opposed to a sequence) in a for loop."

Python-API"
" A class that wants to be an iterator should implement two methods: a 
next() method that behaves as described above, and an __iter__() method 
that returns self." ... "Iterators are currently required to support 
both protocols."


The clear intention is that iterators be usable as iterables.

https://docs.python.org/3/glossary.html
iterator:
" Iterators are required to have an __iter__() method that returns the 
iterator object itself so every iterator is also iterable and may be 
used in most places where other iterables are accepted."



but it may not be *useful* in certain circumstances, and it definitely
may not be *expected* (because nearly all iterators are iterables).
"Broken" is a strong word to use, though, and that might be why the
debate is continuing this long...


I think 'semi-iterator' might be a better term, definitely more neutral, 
for an object that is maybe duck-type usable as an iterator and maybe not.


For Python code, I currently do not see a reason to omit the minimal 
"def __init__(self): return self".  I don't know about C code.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XHGL3BWV5UNZYOEPQLVT7H5YRRO3UQBU/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Should the definition of an "(async) iterator" include __iter__?

2021-09-15 Thread Terry Reedy

On 9/15/2021 12:33 AM, Guido van Rossum wrote:
On Tue, Sep 14, 2021 at 9:03 PM Steven D'Aprano > wrote:


On Tue, Sep 14, 2021 at 12:33:32PM -0700, Guido van Rossum wrote:
 > My view of this is:
 >
 > A. It's not an iterator if it doesn't define `__next__`.
 >
 > B. It is strongly recommended that iterators also define `__iter__`.
 >
 > In "standards" language, I think (A) is MUST and (B) is merely
OUGHT or
 > maybe SHOULD.

That's not what the docs say :-)

https://docs.python.org/3/library/stdtypes.html#iterator-types



Like Steven, I consider 'iterators are iterables' to be a very positive 
feature.


Huh, so it does. And in very clear words as well. I still don't think 
this should be enforced by checks for the presence of __iter__ in 
situations where it's not going to be called (e.g. in iter() itself and 
in "for x in it").


I agree with this also as I consider 'duck typing' (delayed type 
checking by use) and 'consenting adults' (break rules at one's own risk) 
to also be features. If iter were to check for __iter__ on the return 
object, it might as well call it to see if it returns the same object. 
That might be appropriate for a 'SargentPython' implementation, but, to 
me, not for CPython.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DJPR23CY6N3GP6DDULPBMSHWH7JKWMFT/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 467 feedback from the Steering Council

2021-09-09 Thread Terry Reedy

On 9/9/2021 1:56 PM, Ethan Furman wrote:

On 9/9/21 9:37 AM, Barry Warsaw wrote:

 > While I think int.to_bytes() is pretty obscure (I knew about it, 
forgot about it, and learned
 > about it again!) I’m not so sure it’s any less obscure than a 
proposed bytes.fromint().

 >
 > So why don’t we just relax int.to_bytes()’s signature to include 
natural default values:

 >
 >  int.to_bytes(length=1, byteorder=sys.byteorder, *, signed=False)


Default arg values are one of Python's great features.


 > Then I ought to be able to just do
 >
 >  >>> (65).to_bytes()
 >  b’A’

That seems so much worse than

     >>> bchr(65)
     b'A'

;-)


Except that .to_bytes already exists, and arguably should have had such 
defaults from the beginning, making any new function to do the same 
thing superfluous.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/A7OQPRVVE4WPQ7YCIMCGK2AMPXR6GYTB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Should PEP 8 be updated for Python 3 only?

2021-08-25 Thread Terry Reedy

On 8/25/2021 6:48 PM, Steve Holden wrote:
I suspect it's the same motivation that makes us comment out a block of 
code rather than deleting it, even though we know the VCS will let us 
retirive it whenever we want. If I'm wrong it won't be fatal. Or 
surprising ;-)


Kind regards,
Steve


On Wed, Aug 25, 2021 at 8:53 PM Eric V. Smith > wrote:


I think we’re better off removing them. 2.7 is completely
unsupported by us.

Why do you think they should still be kept?

--
Eric

 > On Aug 25, 2021, at 1:32 PM, Mike Miller mailto:python-...@mgmiller.net>> wrote:
 >
 > 
 > How about moving Python 2 notes into a section at the bottom?


I initially thought of this also, for +- the reason Steve gives.  But 
PEP 8 still specifically applies to the stdlib and we only edit 3.6+. 
So I agree with Eric.


We could add (perhaps at the end of the first paragraph) something like 
"(Since the 2.x stdlib is frozen, all 2.x-specific guidelines were 
removed in Sept 2021.)"  Anyone interested could then check git log for 
the last commit before then.  And everyone would know that what follows 
is for 3.x.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HGHRMW7ZCFDN2Q6I7HFOI3UY5JG5F44J/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Is anyone relying on new-bugs-announce/python-bugs-list/bugs.python.org summaries

2021-08-23 Thread Terry Reedy

On 8/23/2021 6:11 PM, Ammar Askar wrote:

Hey everyone,

As part of PEP 588, migrating bugs.python.org issues to Github, there
are two current mailing list related items that need a replacement or
need to be turned down.

1. Weekly summary emails with bug counts and issues from the week,
example: 
https://mail.python.org/archives/list/python-dev@python.org/thread/JRFJ4QH7TR35HFRQWOYPPCGOYRFAXK24/


Essential for my current non-IDLE contributions.  I get this every 
Friday about noon US Eastern and usually use it for a few hours of 
tracker triage later that day and maybe into Saterday.  I usually close 
at least one issue each week, as either rejected or fixed.



2. Emails sent to the new-bugs-announce and python-bugs-list for new
issues and comments to existing issues.

I wanted to quickly gauge if anyone is relying on these mailing
lists/emails or if it's a convenient part of your workflow. Feel free
to chime in here or on the migration issues directly with your
use-case:

1. https://github.com/psf/gh-migration/issues/6
2. https://github.com/psf/gh-migration/issues/7


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3BPOSYMQQ52RA5KMWUZBHJV6DS7CCGWP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: New moderator

2021-08-23 Thread Terry Reedy

On 8/23/2021 4:02 PM, Ethan Furman wrote:

After petitioning the Steering Council I have been added as a moderator 
for the Python Dev mailing list.  I was already the moderator of four 
other Python lists.


Thank you Ethan.  I think you have done better than I would have with 
python-list.


My goal as a moderator is to have our forums be civil and within the 
CoC.  I do rely heavily on users to let me know if someone may be 
crossing the line.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F4ALR5JRKUY3SJ6ZHB4KC7YZGAWJ67X6/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Dropping out of this list

2021-08-19 Thread Terry Reedy

On 8/19/2021 2:47 PM, Marco Sulla wrote:

I think you're writing to me.


I was writing to 3 people including you and everyone in general 
requesting that people be more careful when responding.



I simply clicked the "Reply to all"


Don't do that.  Just followup to the list.


button because I'm lazy.



On Thu, 19 Aug 2021 at 20:38, Terry Reedy  wrote:


As I said before, I am using Ignore Thread to ignore these threads.
Please stop evading my wishes by sending me private discourtesy copies.
   At least 3 people have done that when *not* responding to anything I
wrote.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QO4S64HJIV3AB5TZZKICABELAZYXEXI4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Dropping out of this list

2021-08-19 Thread Terry Reedy
As I said before, I am using Ignore Thread to ignore these threads. 
Please stop evading my wishes by sending me private discourtesy copies. 
 At least 3 people have done that when *not* responding to anything I 
wrote.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KVI3OIM5N5XJLIWFHEMH67GMF6JFJDOT/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Dropping out of this list

2021-08-18 Thread Terry Reedy

On 8/18/2021 9:37 PM, Edwin Zimmerman wrote:

On 8/18/21 9:18 PM, Jonathan Goble wrote:

I am mostly a lurker, but I am also considering unsubscribing if someone 
doesn't step in and stop the mess


+1


Both the email and newsreader parts of Thunderbird have an option called 
Ignore Thread.  Do your readers have such?



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZIPFXQ7UPOJYWBPM2ZSOTMS62CG5FY33/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Problems with dict subclassing performance

2021-08-15 Thread Terry Reedy
SUMMARY: If you, Marco, want to get dicts subclasses made faster and you 
seriously think that they can be, open a proper issue on 
bugs.python.org., as I describe in 3 below.


In any case, drop this tread, which started off wrongly.

August 6, in response to the weekly post,
Summary of Python tracker Issues
Marco wrote this off-topic reply:

> I've done an answer on SO about why subclassing `dict` makes the
> subclass so much slower than `dict`. The answer is interesting:
[link to SO post]
> What do you think about?

What I thought at the time or think now:
1. Starting a new topic in a reply to something else is a  beginner 
mistake.  For one thing, such posts tend to get ignored, as this was.
2. This post has the form of spam intended to drive traffic elsewhere: a 
teaser giving essentially no information and a link to follow to get 
some.  Most posts of this form are filtered out by moderators.  Even 
when passed, such posts tend to be ignored.


3. Pydev, as well as b.p.o., are for improving Python and CPython.  A 
post intended to help do that might look like:


"Subclasses of dict are much slower that dict.

[Note that slower in itself is known and not surprising.]
The reason seems to be ,
possibly followed by the SO link>
"I think dicts subclasses could be made faster by
.

4. Such a post should probably go on b.p.o. unless you think there is a 
policy issue that might prevent a patch.  A pydev post should address 
the issue.


On August 12, you complained about the lack of response
(for which I have given two reasons above)
> No ideas? Excuse me for the up.

"up"? There is no update. A ping after just 4 days is a bit spammy. 
Good thing not everyone does that.


In response Paul Moore asked about a b.p.o. issue, as nothing happens 
without one.  And


On 8/15/2021 10:22 AM, Marco Sulla wrote:
(without quoting his previous message>

On Thu, 12 Aug 2021 at 12:54, Steven D'Aprano  wrote:

Are you looking for upvotes on StackOverflow


A perhaps snarky response to a slightly spammy pair of posts.  You were 
clearly looking for clicks.  Twice.  Since you do not own the site, Why?



This is unacceptable. I pretend your immediate excuses.


As Chris implied, the second 'sentence' is not grammatical English.  I 
can vaguely imagine a couple of things you might mean, but will not 
guess.  But please stop what you started so badly.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/M3JMR5GKWGK2APETJLVMDAKEYKHWVM7K/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Making code object APIs unstable

2021-08-13 Thread Terry Reedy

On 8/13/2021 8:45 PM, Guido van Rossum wrote:

Now, backwards compatibility is still nothing to sneeze at, but at least 
we don't have to hem and haw about ABI compatibility.


If back compatibility were our sacred prime directive, then we would not 
(or should not) have changed code objects in way that threatens it.


The question then is, can we break (source) backwards compatibility for 
these two functions, because in practice code objects have been 
unstable?


I presume that the existence of those functions is based on an 
expectation of stability that may have been true for years, but not 
recently.  Or an expectation that any change would be part of a one-time 
overhaul where compatibility was suspended as much as for 3.0.  I think 
that whatever we do now should allow for continued version-by-version 
changes in the future.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JKHISUP624VI7MVUG7XPYO5QTBLFIUT2/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Making code object APIs unstable

2021-08-13 Thread Terry Reedy

On 8/13/2021 1:24 PM, Guido van Rossum wrote:
In 3.11 we're changing a lot of details about code objects. Part of this 
is the "Faster CPython" work, part of it is other things (e.g. PEP 657 
-- Fine Grained Error Locations in Tracebacks).


As a result, the set of fields of the code object is changing. This is 
fine, the structure is part of the internal API anyway.


But there's a problem with two public API functions, PyCode_New() and 
PyCode_NewWithPosArgs(). As we have them in the main (3.11) branch, 
their signatures are incompatible with previous versions, and they have 
to be since the set of values needed to create a code object is 
different. (The types.CodeType constructor signature is also changed, 
and so is its replace() method, but these aren't part of any stable API).


Unfortunately, PyCode_New() and PyCode_NewWithPosArgs() are part of the 
PEP 387 stable ABI. What should we do?


PEP 387 is Backwards Compatibility Policy
https://www.python.org/dev/peps/pep-0387/
Did you mean PEP 384 -- Defining a Stable ABI
https://www.python.org/dev/peps/pep-0384/

How are PyCode_xxx included?  384 defines code objects as 'internal 
objects'.  "In some cases, the incompatibilities only affect internal 
objects of the interpreter, such as frame or code objects."  And Firefox 
does not find 'pycode'.


A. We could deprecate them, keep (restore) their old signatures, and 
create crippled code objects (no exception table, no endline/column 
tables, qualname defaults to name).


B. We could deprecate them, restore the old signatures, and always raise 
an error when they are called.


These seem pretty useless.


C. We could just delete them.

D. We could keep them, with modified signatures, and to heck with ABI 
compatibility for these two.


E. We could get rid of PyCode_NewWithPosArgs(), update PyCode() to add 
the posonlyargcount (which is the only difference between the two), and 
d*mn the torpedoes.


F. Like (E), but keep PyCode_NewWithPosArgs() as an alias for 
PyCode_New() (and deprecate it).



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3OXTGNT5DTBJBWF7ZXLQB22NYNZWHJI7/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations

2021-08-11 Thread Terry Reedy

On 8/11/2021 7:56 AM, Larry Hastings wrote:

So, here's an idea, credit goes to Eric V. Smith.  What if we tweak how 
decorators work, /jst slghtly/, so that they work like the 
workaround code above?


Specifically: currently, decorators are called just after the function 
or class object is created, before it's bound to a variable.  But we 
could change it so that we first bind the variable to the initial value, 
then call the decorator, then rebind.  That is, this code:


@dekor8
class C:
     ...

would become equivalent to this code:

class C:
     ...
C = dekorate(C)


This is how function decorators were originally defined. Before the 2016 
(3.5) revision of

https://docs.python.org/3/reference/compound_stmts.html#function-definitions
by https://bugs.python.org/issue26576

---
   @f1(arg)
   @f2
   def func(): pass

is equivalent to

   def func(): pass
   func = f1(arg)(f2(func))
---

After
---

@f1(arg)
@f2
def func(): pass

is roughly equivalent to

def func(): pass
func = f1(arg)(f2(func))

except that the original function is not temporarily bound to the name func.
---

I questioned on the issue whether the non-binding optimization "should 
it be documented as a guaranteed language feature or as just an optional 
optimization?"


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PG4UIVT3XEWUS5TVO4MJIQV47UOVV3TD/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: 咨询python的相关问题(Consult python related issues)

2021-07-28 Thread Terry Reedy

On 7/28/2021 6:46 AM, C wrote:

Hello, I would like to consult related issues encountered when reading 
the python language reference manual,


Ask question on python-list, which is about *using* python.
https://mail.python.org/mailman/listinfo/python-list

python-dev is for discussion of *developing* future versions of Python 
and CPython.


Both are English only.

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WKEITIR5S65NSOVWGPV7FPUDXGVHXTBG/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Stale PR

2021-07-06 Thread Terry Reedy

On 7/6/2021 12:29 PM, Brian Curtin wrote:
Before looking at the code, my first question would be about the 
description: "I kinda ran out of time, i suspect more testing is due."


If you were out of time then it's probably not done and maybe lacks the 
tests you initially thought it did, so did you find the time 
Tests were added later the same day.  The opening comment should have 
been edited (I made an attempt at it).


> and/or is the PR done?

Asked on PR.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3UI6TTUZ5A5NPUHNBAEGSYWQSFWBGERK/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Critique of PEP 657

2021-06-30 Thread Terry Reedy

On 6/30/2021 5:30 PM, Pablo Galindo Salgado wrote:
Also, notice we are extending the traceback module (in Python) to 
support this, so you probably can also leverage those changes so you 
don't need to mess with code objects yourself :)


IDLE currently uses traceback.extract_tb and traceback.print_list.  In 
between, it a) removes extra entries at both ends that result from 
running in IDLE, and b) adds code lines for shell entries.  It does this 
in the user code execution process and send the resulting string tagged 
as stderr via the socket connection to the IDLE gui process.


What I believe I would like is to have 'line n' of each frame entry 
replaced with a position 4-tuple, however formatted, and no caret line. 
 IDLE would then use the position to tag the appropriate slice of the line.


Currently, if the user right clicks on either of the two lines of pair, 
to see the line in its context in its file, IDLE opens the file in an 
editor if not open already and  tags the entire line.  If 'line n' were 
replaced with the slice info, it could instead tag that slice, either 
within a line or spanning multiple lines.  Both would be improvements.


Please add me as nosy to any appropriate issues/PRs so I have at least 
an opportunity to test and comment.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/E5NSLCFQBR6M27YZQHRXRQKO657O5GF4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Critique of PEP 657

2021-06-30 Thread Terry Reedy

 > Then how will modules that customizes traceback presentation, such as
idlelib, be able to get the 4-tuple for a particular traceback entry?

 From the exception, you can get the code object and from the code 
object the extra information using

the Python API:

Example:

 >>> try:
...   1/0
... except Exception as e:
...   f = e



 >>> list(f.__traceback__.tb_frame.f_code.co_positions())
[(1, 4, 1, 8), (2, 2, 3, 4), (2, 2, 5, 6), (2, 2, 3, 6), (2, 2, 3, 6), 
(2, 4, 3, 8), (2, 4, 3, 8), (None, 2, 3, 6), (3, 4, 1, 8), (3, 3, 8, 
17), (3, 4, 1, 8), (3, 4, 1, 8), (3, 4, 1, 8), (3, 4, 1, 8), (4, 4, 7, 
8), (4, 4, 3, 4), (4, 4, 3, 8), (4, 4, 3, 8), (4, 4, 3, 8), (4, 4, 3, 
8), (4, 4, 3, 8), (4, 4, 3, 8), (None, 4, 3, 8), (None, 4, 3, 8), (None, 
4, 3, 8), (None, 4, 3, 8), (None, 4, 3, 8), (3, 4, 3, 8)]


Ah, co_positions is an access method, corresponding to the current (also 
undocumented) co_lines.  There will be no direct access to the 
position-table as it will be kept private and subject to change.  The 
obvious first question: why 28 items and what does the index mean?


When I compile the above with 3.10.0b3, there are 29 bytecodes, so I am 
guessing your branch has 28 and that the first number in the tuple is 
the line number.  But how would one know which '2' entry corresponds to 
the divide in '1/0'.  And what do the rest of the tuple numbers mean?  I 
don't see anything like the (2,2, 2,4) I expect for '1/0'.  To be 
documented, as you say below.


This is equivalent (and more efficient) to have this information in the 
traceback itself (as it would have been duplicated and would require 
more changes).


Understood and agreed.

We would document this a bit better with some examples. And we will make 
sure to add docs about this for sure :)



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/27NHADGIRL2KYU42LACX445TW53ENKWL/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Critique of PEP 657

2021-06-30 Thread Terry Reedy

On 6/30/2021 12:30 PM, Ammar Askar wrote:


I don't think we're making strong claims that the full `(line,
end_line, column, end_column)` should be the canonical representation
for exception locations. The only concrete place we suggest their
usage is in the printing of tracebacks.


sys.__excepthook__ will have access to the information, but will censor 
it for long lines or slices that span more than one line.



The information is not exposed
in the exception or traceback objects intentionally as part of this.


Then how will modules that customizes traceback presentation, such as 
idlelib, be able to get the 4-tuple for a particular traceback entry? 
This seems like a repeat of attribute and name error name hints being 
not accessible from the traceback module and only accessible from Python 
via a workaround such as in idlelib.run: 218:


if typ in (AttributeError, NameError):
# 3.10+ hints are not directly accessible from python (#44026).
err = io.StringIO()
with contextlib.redirect_stderr(err):
sys.__excepthook__(typ, exc, tb)
return [err.getvalue().split("\n")[-2] + "\n"]

As Pablo explained on #44026, the length hints are not part of the tb 
object because they are expensive to compute and are not useful when 
AttributeError and NameError are expected and are caught in code for 
flow control purposes.  Therefore the hints are only computed when an 
uncaught exception is displayed to users.


However, the position information is already computed and it is just a 
matter of passing it along *all the way* to the python coder.


For slices within normal lines, the new caret line can be turned back 
into position, as IDLE does now for SyntaxErrors.  But that does not 
work when the caret represents a truncated slice.


The PEP says co_positions will be added code objects but makes no 
mention of an API for accessing information within it.  And that still 
leaves the issue of getting the code object.


Summary: the position information would be much more useful if it were 
added to traceback items and if the traceback functions then exposed it.


Note: the current co_lines and co_linetable seems to be undocumented -- 
no index entries, nothing in

https://docs.python.org/3.11/reference/datamodel.html#index-56
and no docstring for co_lines.  So I have no idea how these work.

Note 2: "Opt-out mechanism" says new environmental variable, new 
-Xnodebugranges flag. "Have a configure flag to opt out" says "we have 
decided to the -O flag".  I presume the latter is obsolete.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RQYVQXJXZRLCWXVXKICJXB2RQLL2ZEXM/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: [python-committers] Re: Roundup to GitHub Issues migration

2021-06-23 Thread Terry Reedy

On 6/23/2021 4:28 PM, Inada Naoki wrote:

FWIW, GitHub announced new powerful Issues today.
https://github.com/features/issues

It may fill some gap between GitHub Issues and Roundup.


I signed up for the beta waiting list so I could experiment with it. 
Someone else would have to sign up 'Python'.


Terry

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DZGIYHVP5XSF5EVQZP2DZVQXOTB2O47G/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Roundup to GitHub Issues migration

2021-06-22 Thread Terry Reedy

On 6/22/2021 3:52 PM, Brett Cannon wrote:

One thing I will remind people is I personally have led the work to move 
this project from:


 1. SourceForge to our own infrastructure
 2. Mercurial to git
 3. Our own infrastructure to GitHub for code management


At this point, I (once a skeptic) agree that the migration is an overall 
improvement, with CI being the roughest.


As for issue migration: github PRs are much 'richer' in info than 
Rietveld reviews, and there is now duplication of information with bpo 
issues.  (Where synchonization is not automated, this can be a nuisance.)


This means to me that the github issue metadata can be simplified. 
After making the migration easier, this will mean less bad metadata to 
be corrected by triagers.


With 2.x gone and a backport flow, essentially all issues are for main, 
so leave version tags off the issue (and eliminate the need to update 
them!).  [3.x] in PR titles and backport labels are enough.  Backport 
labels imply 'behavior' versus 'enhancement.


Stage info is also on the PR.

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RM2ETTE2TNSC6W7QCG2HZHW4D72IWUGU/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Roundup to GitHub Issues migration

2021-06-22 Thread Terry Reedy

On 6/22/2021 4:40 AM, Baptiste Carvello wrote:


Le 21/06/2021 à 23:31, Christopher Barker a écrit :

Also: cPython is a large, complex, and mature project. I don't think
many non-developers can even identify a true bug, much less write a
helpful big report. [...]



There is a genuine question here: bluntly said, are bug reports still
welcome? 


The above is one person's viewpoint.  Since he is not a CPython core 
developer, I am not sure what he means by 'developer'.  Anyone who 
programs in Python 'developes' the programs they write.


It is true that students and beginners tend to make less useful reports. 
 But yes, bug reports are still welcome.



In the tradition of the Free Software movement, reporting bugs
is considered an act of good citizenship. If the scarcity of developer
time makes it no longer be the case for cPython, it'd rather be broadly
communicated to users.


What we need is for people to be less shy about helping with *other* 
people's reports.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GPGYOVA5FXPWQBDZWGBU4BVNFMKBDGOZ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Roundup to GitHub Issues migration

2021-06-22 Thread Terry Reedy

On 6/22/2021 4:00 AM, Tiziano Zito wrote:

Hi,

On Mon 21 Jun, 13:48 +0200, Victor Stinner  wrote:

The requirement for a GitHub account was well known when PEP 581 was
accepted. The PEP was approved. It's now time to move on!


I think it is important to notice that GitHub actively blocks user 
registration and activity from countries that are sanctioned by the US 
government. At least in 2019 GitHub was blocking users from IPs located 
in Cuba, North Corea, Syria, Crimea, Iran, etc (see for example [1]). 
They block, of course, users of any nationality, if they happen to be 
traveling or living in those countries.



[1] https://techcrunch.com/2019/07/29/github-ban-sanctioned-countries/


It says that this applied to private repositories and paid organization 
repositories but not public, open source repositories.  I don't know if 
the forks of python needed to submit PRs are affected, but I belive 
these are both free and public.


https://docs.github.com/en/github/site-policy/github-and-trade-controls

Says that Github has an explicit license to serve Iran and most services 
are available in Cuba.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TTAZO263HP2GCHE2KRTWLI2FDTC2AG4C/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: My help with yours IDLE

2021-06-07 Thread Terry Reedy

On 6/7/2021 12:20 PM, MatroCholo wrote:

Dear Python Developers,
I have found that default IDLE can't open .py files on double click


After installing with the python.org Windows installer, double-clicking 
x.py should run x.py with the default python.  It does for me.  The file 
should be designed for this to make much sense.



and IDLE isn't shown in "Open with" menu.


It should be shown in the Edit with IDLE menu.  For me, 3.10 is, 3.9 is 
not.  There might be a tracker issue for this.  IDLE is a development 
and learning environment, so running from the editor is usually the best 
way to run in Python.


I have solved this problem by 
converting idle.bat in \lib\idlelibs\ into .exe file.


The fact that one can only 'open' (run) with a .exe and not a .bat file 
has come up on a bugs.python.org tracker issue.


If you are interested in my help, you can learn more in my GitHub repo - 
https://github.com/MatroCholo/idle2exe 


I don't read Russian, so the readme is not useful to me.

If you want to discuss IDLE development more, in English, post on the 
idle-dev mailing list.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UZK2M7MZTCB5ZK5OSVTVQRD2POV37RKP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

2021-05-20 Thread Terry Reedy

On 5/20/2021 10:49 AM, Oscar Benjamin wrote:

On Thu, 20 May 2021 at 04:58, Terry Reedy  wrote:



I believe the ratio for the sort of numerical computing getting bogus
complaints is sometimes more like 95% of *time* in compiled C and only,
say, 5% of *time* in the Python interpreter.  So even if the interpreter
ran instantly, it would make also no difference -- for such applications.


'also' was meant to be 'almost'


Not necessarily


In the context I carefully defined, where Python is falsely accused of 
endangering the earth, by people who set up strawpeople images of how 
Python is actually used and who care nothing about programmer time and 
joy, yes, necessarily.  However, in the related context you define, 
faster Python could help save the earth by reducing the need for 
brute-force C routines when they are grossly inefficient.  How ironic 
that would be.



because if the interpreter is faster then it opens up
new options that perhaps don't involve the same C routines. The
situation right now is that it is often faster to do more
"computation" than needed using efficient C routines rather than do
precisely what is needed in bare Python. If the bare Python part
becomes faster then maybe you don't need the C routine at all.

To give a concrete example, in SymPy I have written a pure Python
implementation of typed sparse matrices (this is much faster than the
public Matrix class so don't compare with that). I would like to use
the flint library to speed up some of these matrix calculations and
the flint library has a highly optimised C/assembly implementation of
dense matrices of arbitrary precision integers. Which of these
implementations is faster for e.g. matrix multiplication depends on
how sparse the matrix actually is. If I have a large matrix say 1000 x
1000 and only 1% of the elements are nonzero then the pure Python
sparse implementation is faster (it can be much faster as the density
reduces since it does not have the same big-O characteristics). On the
other hand for fully dense matrices where all elements are nonzero the
flint implementation is consistently around 100x faster.

The break even point where both implementations take equal time is
around about 5% density. What that means is that for a 1000 x 1000
matrix with 10% of elements nonzero it is faster to ask flint to
construct an enormous dense matrix and perform a huge number of
arithmetic operations (mostly involving zeros) than it is to use a
pure Python implementation that has more favourable asymptotic
complexity and theoretically computes the result with 100x fewer
arithmetic "operations". In this situation there is a sliding scale
where the faster the Python interpreter gets the less often you
benefit from calling the C routine in the first place.

Although this is a very specific example it illustrates something that
I see very often which is that while the efficient C routines can make
things "run at the speed of C" you can often end up optimising things
to use an approach that would seem inefficient if you were working in
C directly. This happens because it works out faster from the
perspective of pure Python code that is encumbered by interpreter
overhead and has a limited range of C routines to choose from. If the
interpreter overhead is less then the options to choose from are
improved.

Also for many applications it is much easier for the programmer to
write an algorithm directly in loops rather than coming up with a
vectorised version based on e.g. numpy arrays. Vectorisation as a way
of optimising code is actually work for the programmer. There is
another tradeoff here which is not about C speed vs Python speed but
about programmer time vs CPU time. If a straight-forward Python
implementation is already "fast enough" then you don't need to spend
time thinking about how to translate that into something that would
possibly run faster (potentially at the expense of code readability).
In the case of SymPy/flint if the maximum speed gain of flint was only
10x then I might not bother using it at all to avoid the complexity of
having multiple implementations to choose from and external
dependencies etc.

--
Oscar




--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PLPOOWB6KRBSIBZLM5NOP2O5AJGAN2AB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

2021-05-19 Thread Terry Reedy

On 5/13/2021 4:18 AM, Mark Shannon wrote:

Hi Terry,

On 13/05/2021 5:32 am, Terry Reedy wrote:

On 5/12/2021 1:40 PM, Mark Shannon wrote:

This is an informational PEP about a key part of our plan to improve 
CPython performance for 3.11 and beyond.



As always, comments and suggestions are welcome.


The claim that starts the Motivation section, "Python is widely 
acknowledged as slow.", has multiple problems. While some people 
believe, or at least claim to believe "Python is slow", other know 
that as stated, the latter is false.  Languages do not have a speed, 
only implementations running code for particular applications have a 
speed, or a speed relative to equivalent code in another language with 
a different runtime.


I broadly agree, but CPython is largely synonymous with Python and 
CPython is slower than it could be.


The phrase was not meant to upset anyone.
How would you rephrase it, bearing in mind that needs to be short?


Others have given some fine suggestions.  Take your pick.

[ship]
We want people to be able to write code in Python and have it perform at 
the level they would get from a good Javascript or lua implementation.


I agree with others that this is a good way to state the goal.  It also 
seems on the face of it reasonable, though not trivial.  I get the 
impression that you are proposing to use python-adapted variations of 
techniques already used for such implementations.




It is still important to speed up Python though.


I completely agree.  Some application areas are amenable to speedup be 
resorting to C libraries, often already available.  Others are not.  The 
latter involve lots of idiosyncratic business logic, individual numbers 
rather than arrays of numbers, and strings.


Numpy based applications gain firstly from using unboxed arrays of 
machine ints and floats instead of lists (and lists) of boxed ints and 
floats and secondly from C or assembly-coded routines


Python strings are already arrays of machine ints (codepoints).  Basic 
operations on strings, such as 'substring in string' are already coded 
in C working on machine values.  So the low-hanging fruit has already 
been picked.


If a program does 95% of its work in a C++ library and 5% in Python, it 
can easily spend the majority of its time in Python because CPython is a 
lot slower than C++ (in general).


I believe the ratio for the sort of numerical computing getting bogus 
complaints is sometimes more like 95% of *time* in compiled C and only, 
say, 5% of *time* in the Python interpreter.  So even if the interpreter 
ran instantly, it would make also no difference -- for such applications.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GFUISO3ZDYS7E3FV457AVRW2Q7B5BAVW/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

2021-05-13 Thread Terry Reedy

On 5/12/2021 1:40 PM, Mark Shannon wrote:

This is an informational PEP about a key part of our plan to improve 
CPython performance for 3.11 and beyond.


What is the purpose of this PEP?  It seems in part to be like a 
Standards Track PEP in that it proposes a new (revised) implementation 
idea for the CPython bycode interpreter.  Do you not intend this to not 
constitute approval of even the principle?


One of the issues in the new project gave formulas for the cost versus 
benefit calculations underlying specialization.  Depending on your 
purpose, it might be good to include them.  They certainly gave some 
clarity to me.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I3UXYIMASCD5MDJ5QU6ITF6CRKIURHQ5/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 659: Specializing Adaptive Interpreter

2021-05-12 Thread Terry Reedy

On 5/12/2021 1:40 PM, Mark Shannon wrote:

This is an informational PEP about a key part of our plan to improve 
CPython performance for 3.11 and beyond.



As always, comments and suggestions are welcome.


The claim that starts the Motivation section, "Python is widely 
acknowledged as slow.", has multiple problems. While some people 
believe, or at least claim to believe "Python is slow", other know that 
as stated, the latter is false.  Languages do not have a speed, only 
implementations running code for particular applications have a speed, 
or a speed relative to equivalent code in another language with a 
different runtime.


I reason I am picking on this is that the meme 'Python is slow' is being 
morphed into 'Python is destroying the earth' (and should be abandoned, 
if not banned).  Last fall, a science news journal (Nature News?) quoted 
a 'concerned scientist' saying just this.  An internet troll repeated it 
last week on comp.lang.python (from where it leaked onto python-list).


It is true that Python has characteristics that make it *relatively* 
difficult to write interpreters that are *relatively* fast in certain 
applications.  But the opposite is also true.  The language does *not* 
mandate that objects, their methods, and modules be written in the language.


Hence, CPython implements builtin objects and function and some stdlib 
modules in C and allows 3rd party modules written in C or C++ or 
Fortran. I believe the first killer app for Python, in the mid 1990s, 
numerical computing with NumericalPython.  Rather than being 'slow', 
CPython *enabled* people, with a few percent of added time, to access 
fast, heavily optimized C and Fortran libraries and do things they could 
not do in Fortran and that would have been much more difficult in C.  My 
daughter's PhD thesis work is a recent example of using Python to access 
C libraries.


The concerned scientist mentioned above noted, more or less correctly, 
that numerical code, such as neuro-network code, is, say, 80x slower in 
pure python than in compiled C.  But he did not mention that serious 
numerical and scientific work in Python is not done with such code.

I have seen this sort of bogus comparison before.


https://www.python.org/dev/peps/pep-0659/


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/M76T6V7UXJRARMVM2CQOMEIOOB4FQZQZ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Speeding up CPython

2021-05-12 Thread Terry Reedy

On 5/12/2021 5:14 PM, Antoine Pitrou wrote:

On Wed, 12 May 2021 17:05:03 -0400
Terry Reedy  wrote:



Yet you always see it: new people not knowing where to start,
highly skilled contributors drowning and
intermediate contributors moving slowly


I have multiple times strongly recommended that people review issues and
PRs, and sometimes given details, but most won't or don't.


I don't know who "people" are in your sentence, but reviewing issues
and PRs generally requires a high familiarity with a project, and


Much can be done without what I think you mean by 'high familiarity'.

Bug issues:
  bpo: "On macOS with 3.8.3 I see this buggy behavior" If not enough 
info to reproduce, ask for it.  If there is, try to reproduce on lastest 
release or even better, repository build.  Sometimes, trying on a 
different OS is helpful.

  PR: make local PR branch and test whether proposed fix works.

Enhancement issues:
  bpo: if proposal is for core python or a module one has used, does 
proposal seem like an actual improvement?  enough to be worth the likely 
bother?

  PR: does the PR work as promised?  Do you like it?

PR Python code: read it.  See any possible improvements?


enough confidence to speak with a voice of (seeming) authority.


I prefer honesty to pretend authority.  Nearly 2 years ago, a 'new 
contributor' commented on a IDLE PR, "Would x be better?" It was a minor 
improvement, but a real one, so I made it, thanked the person, and 
encouraged further efforts.  That person did so and is now a core developer.


I would welcome more eyes on IDLE patches and use testing thereof.


I'm
not convinced it's generally easier than submitting a patch for a
particular issue you're comfortable with.


Some review actions are easier, some are not.  But the only way to learn 
to review other people's code is to do it, and it is a skill we need in 
at least some new coredevs.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EKUEY6FZNUNYLJB2RDMH7JMLBIQ2FQXO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Speeding up CPython

2021-05-12 Thread Terry Reedy

On 5/12/2021 2:50 PM, Abdur-Rahmaan Janhangeer wrote:

Great news, just a tiny bit from me.
I read the other day in the OpenSource report
sponsored by the Ford Foundation a CPython
contributor stating that we have an all time high
count of Python users but an all time low number of
contributors to CPython. I don't know how but
we certainly need a fake path to help people start


I presume you mean 'fast path'?


contributing and level up to gain a pool of resources
We don't need to wait for easy issues or things like
that or wait for PR merge to level up.

Yet you always see it: new people not knowing where to start,
highly skilled contributors drowning and
intermediate contributors moving slowly


I have multiple times strongly recommended that people review issues and 
PRs, and sometimes given details, but most won't or don't.  Do you have 
any idea why?


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TXDU2Q4D2BA2MGHXBGDP4VI452NSCVDB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-10 Thread Terry Reedy

On 5/10/2021 6:07 AM, Steven D'Aprano wrote:

On Mon, May 10, 2021 at 05:34:12AM -0400, Terry Reedy wrote:

On 5/10/2021 3:28 AM, M.-A. Lemburg wrote:


I'm mostly thinking of tracebacks which go >10 levels deep, which is
rather common in larger applications. For those tracebacks, the top
entries are mostly noise you never look at when debugging. The proposal
now adds another 10 extra lines to jump over :-)


If the slice were instead marked with color tagging, as I hope will be
possible in IDLE and other IDEs, then no extra lines well be needed


That's great for people using IDLE, but for those using the vanilla
Python interpreter, M-A.L makes a good point about increasing the
vertical size of the traceback which will almost always be ignored.


The vanilla interpreter could be updated to recognize when it is running 
on a similated 35-year-old terminal that implements ansi-vt100 color 
codes rather than a similated 40+-year-old black-and-white teletype-like 
terminal.


Making the enhancement available to nonstandard python-coded interfaces 
is a separate issue.



Its especially the case for beginners. Its hard enough to get newbies to
read *any* of the traceback. Anything which increases the visual noise
of that is going to make it harder.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MX5MNQN5WFETYXDE3OK5X6I7BEGUQTQ3/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

2021-05-10 Thread Terry Reedy

On 5/10/2021 3:28 AM, M.-A. Lemburg wrote:


I'm mostly thinking of tracebacks which go >10 levels deep, which is
rather common in larger applications. For those tracebacks, the top
entries are mostly noise you never look at when debugging. The proposal
now adds another 10 extra lines to jump over :-)


If the slice were instead marked with color tagging, as I hope will be 
possible in IDLE and other IDEs, then no extra lines well be needed


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RPAFKIZOC2LQY57S4TK4P5O527RGCGPK/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: On the migration from master to main

2021-05-03 Thread Terry Reedy

On 5/3/2021 9:27 PM, Terry Reedy wrote:

On 5/3/2021 7:45 PM, Tim Peters wrote:

I'm guessing it's time to fiddle local CPython clones to account for
master->main renaming now?



Blob 2 ("upstream"):

"""
The CPython repository's default branch was renamed from ``master`` to 



``main``
after the Python 3.10b1 release. If you had cloned the repository 
before this

change, you can rename your local branch as follows::

 git branch -m master main
 git fetch upstream
 git branch -u upstream/main main
"""

 From my dim understanding, "upstream" makes more sense, but I don't
know.


For 'pull from upsteam, push to origin' workflow,
https://docs.github.com/en/github/administering-a-repository/renaming-a-branch 



says the 2nd, +
git remote set-head origin -a

+ optionally, 'to remove tracking references to the old branch name'
git remote prone origin

When I did this, there was a '[pruned]' line for every branch ever on 
the fork (about 200 for me), including the bots temporary backport 
branches.


I am now preparing a small PR and will test if creation works as usual.


It does.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HTIREIDBUIFYP5YDMARDBGTBL3MSMYEA/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: On the migration from master to main

2021-05-03 Thread Terry Reedy

On 5/3/2021 7:45 PM, Tim Peters wrote:

I'm guessing it's time to fiddle local CPython clones to account for
master->main renaming now?

If so, I've seen two blobs of instructions, which are very similar but
not identical:

Blob 1 ("origin"):

"""
You just need to update your local clone after the branch name changes.
 From the local clone of the repository on a computer,
run the following commands to update the name of the default branch.

$ git branch -m master main
$ git fetch origin
$ git branch -u origin/main main

Apart from that, you should update any local script or command that uses
the name "master" to use the name "main".
"""

Blob 2 ("upstream"):

"""
The CPython repository's default branch was renamed from ``master`` to ``main``
after the Python 3.10b1 release. If you had cloned the repository before this
change, you can rename your local branch as follows::

 git branch -m master main
 git fetch upstream
 git branch -u upstream/main main
"""

 From my dim understanding, "upstream" makes more sense, but I don't
know.


For 'pull from upsteam, push to origin' workflow,
https://docs.github.com/en/github/administering-a-repository/renaming-a-branch

says the 2nd, +
git remote set-head origin -a

+ optionally, 'to remove tracking references to the old branch name'
git remote prone origin

When I did this, there was a '[pruned]' line for every branch ever on 
the fork (about 200 for me), including the bots temporary backport branches.


I am now preparing a small PR and will test if creation works as usual.
--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CWXHEKLX3ZDJFOPA3ALQRPJ3DN7NM6A6/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: expanduser('~other') reliability and future

2021-04-28 Thread Terry Reedy

On 4/28/2021 1:05 PM, Guido van Rossum wrote:
On UNIX-oid platforms (e.g. BSD, Linux, Mac), ~user/ should be 
reasonably reliable, after all the shell does it. On Windows, only ~/ 
can be relied upon -- the rest is "best effort". I'd be okay with 
deprecating ~user/ on Windows, but on UNIX-oid it should not be 
deprecated IMO.
On all systems, IDLE uses expanduser('~') and expanduser(path).  On Mac, 
it uses
expanduser('~/Library/Preferences/.GlobalPreferences.plist').  So I 
presume deprecating ~something only on Windows should not affect IDLE.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZWXDZ23TUXFRB4WFTCC6LRVK36CJO4XR/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Keeping Python a Duck Typed Language.

2021-04-22 Thread Terry Reedy

On 4/22/2021 9:15 AM, Paul Moore wrote:


Absolutely, I see no problem with "use duck typing for this argument"
being opt-in.


As in x: 'duck'? or x: '!', where '!' means
 'infer it!', or

from typing import Infer
... x: Infer
?

Ditto for ->  ?

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NSALGTJQIITQO3ZHI7E5G2J77JQERAKZ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 563 and 649: The Great Compromise

2021-04-18 Thread Terry Reedy

On 4/17/2021 11:43 PM, Larry Hastings wrote:


The heart of the debate between PEPs 563 and 649 is the question: what 
should an annotation be?  Should it be a string or a Python value?  It 
seems people who are pro-PEP 563 want it to be a string, and people who 



are pro-PEP 649 want it to be a value.

Actually, let me amend that slightly.  Most people who are pro-PEP 
563 
don't actually care that annotations are strings, per se.  What they 
want are specific runtime behaviors, and they get those behaviors when 
PEP 563 turns their annotations into strings.


I have an idea--a rough proposal--on how we can mix together aspects of 


PEP 563 and PEP 649.  I think it satisfies everyone's use cases for both 
PEPs.  The behavior it gets us:


  * annotations can be stored as strings
  * annotations stored as strings can be examined as strings
  * annotations can be examined as values


I agree that somehow satisfying more people rather than fewer is a good 
goal.



The idea:

We add a new type of compile-time flag, akin to a "from __future__" 
import, but not from the future.  Let's not call it "from __present__", 
for now how about "from __behavior__".


I have wondered whether the current split over annotations constitutes 
sufficient 'necessity' to introduce a new entity.  The proposal for 
permanent Python dialects, as opposed to temporary dialects, is not new, 
as indicated by your reference to '__present__'.  It has so far been 
rejected.


Some people, upon having their syntax proposal rejected, have proposed 
that it be made an option.  I believe that others, not liking a accepted 
change, have proposed that the old way be kept as an option, or that the 
new way be optional.  So I think that we should first look at other ways 
to meet the goal, as proposed in other responses.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DBPHCA3MYK3W7GWLE6OKZJCDTQFJJ536/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Revive PEP 396 -- Module Version Numbers ?

2021-04-15 Thread Terry Reedy

On 4/15/2021 12:35 PM, David Mertz wrote:


re 2.2.1


re.__version__ was last modified 2001-10-07 by F. Lundh in 
bec95b9d8825b39cff46a8c645fa0eeb8409854e.  What does '2.2.1' mean?  Only 
Fredrik knows.  The commit message says that he rewrote the pattern.sub 
and pattern.subn methods in C.  That should have been user invisible 
except for speed.


I believe that it is dead data left only for back compatibility and 
because of an inability to give a deprecation warning.



re PackageNotFoundError('re')


It seems like (somehow or another) importlib.metadata is doing something 
perhaps more useful for vaex.  But it is doing distinctly worse for re, 


Is an effectively meaningless and misleading value better than none?

IDLE once had a version last set, I believe, in 2.5 or 2.6, by K. 
Kaiser, with whatever meaning it had to him.  I removed it in 3.6 
(allowed by PEP 434 + RM decision). No one complained.


The real version of any stdlib module is the Python version it is 
released with, plus and external app versions it is associated with 
(sqlite and ssl, for instance).  It has been suggested, if not decided, 
that other versions should go.



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4AA7I3RS6D3UDIBZ6ELTEEWXKXIGE5WS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Revive PEP 396 -- Module Version Numbers ?

2021-04-15 Thread Terry Reedy

On 4/15/2021 12:38 PM, Barry Warsaw wrote:

On Apr 14, 2021, at 23:11, Christopher Barker  wrote:


You wrote the original PEP, so of course you can withdraw it (or reject it), 
but...

Are you sure? See this discussion, I don't think it's as simple as all 

that.


 From a library maintainers point of view, I personally want to get away from 
using __version__ strings at all.  They’re kind of a pain to remember to bump 
on every new release.


At least some in the stdlib have not been.  For re.__version__, not 
since Nov 2001.



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3UNG6KD3JIBHSMWILZGGIFHRSQV6PDKG/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: gzip.py: allow deterministic compression (without time stamp)

2021-04-14 Thread Terry Reedy

On 4/14/2021 8:00 AM, Joachim Wuttke wrote:


Furthermore, if policy about API changes allows, I'd suggest
that `NO_TIMESTAMP` become the new default value for `mtime`.


Changing defaults is a huge pain, which we mostly avoid.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3LA5EIND4O2JGF5SBQBWF3K3W2YPG3EG/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Making staticmethod callable, any oposite?

2021-04-13 Thread Terry Reedy

On 4/13/2021 9:20 PM, Inada Naoki wrote:


But Mark Shannon said we shouldn't make such a change without
discussing at python-dev.
I don't know we *should*, but I agree that it is *ideal*.


I consider this case borderline.  A lot of changes get made, and must 
be, without pydev discussion.


Then, does anyone oppose this change?

Histrically, this idea had been rejected once. bpo-20309 proposed
making classmethod and staticmethod callable.
https://bugs.python.org/issue20309

It had been rejected by:

"I don't agree that this is a bug that should be fixed.  It adds code
that will likely never get called or needed (i.e. there has never been
a request for this in the decade long history of desciptors and it
seems like a made up requirement to me.  "
https://bugs.python.org/issue20309#msg240843


Written by Raymond Hettinger, who continued "If someone object to 
recommendation to close and really wants to push for this, I recommend 
making a business case for acceptance and then assigning this issue to 
Guido for a decision.  This is his code and AFAICT he intentionally 
didn't go down a number of possible paths for descriptors simply because 
there weren't motivating use cases."


You made the case on the issue, and have here, and Guido decided.


"actually supporting this would mean adding code that would need to be
maintained indefinitely without providing a compensating practical
benefit,"
https://bugs.python.org/issue20309#msg240898


Nick Coughlin, following Raymond, who continued
"Thanks Christian for nudging us to make a decision one way or the other."
and
"If another implementation requests clarification, we might want to 
document that "directly callable-or-not" for these descriptors is 
formally an interpreter implementation detail"


So both describe rejection as a close call that could go have gone and 
might in the future go the other way.



But status is changed now. We already have OpenWrapper. It proves
callable classmethod is "called and needed".
Although there is only one use case, we can remove more code than adding.

staticmethod.__call__() is simple C function.
https://github.com/python/cpython/pull/25117/files#diff-57bc77178b3d6f1010dd924722c87522f224d93bc341f0e46c0945094124d8f2

Victor removed OpenWrapper class already, and we can remove `DocDescripter` too.
https://github.com/python/cpython/pull/25354/files#diff-bcdfa9cbb0764d7959cda48f9084d79785f87c5ad7460f27ba2678b0bda76e38R314-L327

I think maintenance burden of staticmethod.__call__() is not higher
than OpenWrapper and DocDescripter.
Additionally, if we have same issue in other module, we can just use
staticmethod, instead of copy OpenWrapper and DocDescripter.

So it provides "compensating practical benefit".



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UAN25M6F45XWNSDRUR4WGT2U5REQMUU4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 649: Deferred Evaluation Of Annotations Using Descriptors, round 2

2021-04-13 Thread Terry Reedy

On 4/13/2021 4:21 AM, Baptiste Carvello wrote:

Le 12/04/2021 à 03:55, Larry Hastings a écrit :



* in section "Interactive REPL Shell":


For the sake of simplicity, in this case we forego delayed evaluation.


The intention of the code + codeop modules is that people should be able 
to write interactive consoles that simulate the standard REPL.  For example:


Python 3.10.0a7+ (heads/master-dirty:a9cf69df2e, Apr 12 2021, 15:36:39) 
[MSC v.1900 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
>>> import code
>>> code.interact()
Python 3.10.0a7+ (heads/master-dirty:a9cf69df2e, Apr 12 2021, 15:36:39) 
[MSC v.1900 64 bit (AMD64)] on win32

Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> # Call has not returned.  Prompt is from code.InteractiveConsole.
>>> def f(x:int): -> float

>>> f.__annotations__ # should match REPL result

>>> ^Z

now exiting InteractiveConsole...
>>> Now back to repl

If the REPL compiles with "mode='single' and spec is changes to "when 
mode is 'single'", then above should work.  Larry, please test with your 
proposed implementation.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KDFTZQMJRBRDBSUSVTMBDXTLCXIZXBFP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Immutable view classes - inherit from dict or from Mapping?

2021-04-12 Thread Terry Reedy

On 4/12/2021 11:29 AM, Andreas R Maier wrote:


I have written ...
My question is, should ...


The pydev list is for development of future Python versions and CPython 
releases.  Questions about the use of, or development with, current 
versions and releases should be directed elsewhere, such as python-list 
or stackoverflow.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/P36Q3LSDGDUCAXUVET5OQ53X42OD22F4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: [EXTERNAL] PEP 647 Accepted

2021-04-10 Thread Terry Reedy

On 4/10/2021 1:02 PM, Guido van Rossum wrote:

I propose that we just clarify this in the docs we'll write for TypeGuard.


I agree.  When I  reviewed the PEP, my concern was not with 'TypeGuard' 
itself, once I understood more or less what it means, but with the 
explanation in the PEP.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NSFP5LWAV4XEYRLXRDQJ2PHEQBKIPWDL/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Help to Resolve issues with Pull request 25220

2021-04-07 Thread Terry Reedy

On 4/7/2021 12:32 PM, Barney Gale wrote:
It looks like you’ve incorporated several other changes into your commit 
by mistake.
The PR definitely has too many changes unrelated to the issue.  I 
recognize a few of the changes as related to recent merges.


  My guess that the the issue-43737 branch was branched off of 
something other than master, which is usually a fatal mistake.  The 
parent branch may also have been branched off of something other than 
master.


Tony, with rare exception, each branch should be branched directly off 
master, which means checking out master first each time.


There is probably some other bad sequence of events that could cause the 
issue.


 The simplest thing to do might be to re-create your git

branch and PR from scratch.


I agree with this.  That means updating the master branch, creating a 
new 43737 branch, make the 43737 changes, commit, and push.


Or, in case problem is elsewise, fix the merge conflict (below), merge, 
and push to your fork.  Update your master from upstream, change to 
branch, git merge upstream/master, commit to branch, push branch to 
fork, and see how things work.


If conflicting changes land while your PR is still open, you’ll 
need to 

do something called a “rebase”.


Rebasing (perhaps done incorrectly) often makes things worse.  One fixes 
a merge conflix by editing the PR branch file(s) with the conflict and 
committing a change.


However, the merge conflict in expression.rst is this:

<<< issue-43737
.. productionlist::
   lambda_expr: "lambda" [`parameter_list`]: `expression`
   lambda_expr_nocond: "lambda" [`parameter_list`]: `expression_nocond`
===
.. productionlist:: python-grammar
   lambda_expr: "lambda" [`parameter_list`] ":" `expression`
>>> master

Adding a grammar production has nothing to do with the issue.  This 
could be easily fixed by removing the lambda_expr_nocond line (and other 
junk if not editing in the local branch) but would not solve the more 
extensive issues.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2XS2INIFALE75X4DQZNO7ZJD5IZVCHEO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-04 Thread Terry Reedy

On 4/4/2021 9:57 AM, Łukasz Langa wrote:


On 4 Apr 2021, at 11:34, Matthias Klose > wrote:



I always tested the release candidates, and built them for various Linux
architectures.  And I'm filing issues marked as 'released-blocker' 
when I see regressions introduced, it's up to the release manager >> to determine  if such changes are intended or not.



Can you show us an example of a release-blocker issue you reported?


Searching creator: doko; priority: release blocker returns 19 issues, 
from 3 to 20 years ago.  That about 1/years but none for 3.8 or 3.9, for 
which Łukasz is RM.


32232 	36 months ago 	building extensions as builtins is broken in 3.7 
has patch 	has PR 	closed
32233 	40 months ago 	[3.7 Regression] build with --with-system-libmpdec 
is broken 	has patch 	has PR 	closed
31016 	43 months ago 	[Regression] sphinx shows an EOF error when using 
python2.7 from the trunk 			closed
23968 	55 months ago 	rename the platform directory from plat-$(MACHDEP) 
to plat-$(PLATFORM_TRIPLET) 	has patch 		closed
26839 	58 months ago 	Python 3.5 running on Linux kernel 3.17+ can block 
at startup or on importing the random module on getrandom() 	has 
patch 		closed
24226 	71 months ago 	[3.5 Regression] unable to byte-compile the 
attached IN.py 			closed
24162 	71 months ago 	[2.7 regression] test_asynchat test failure on 
i586-linux-gnu 			closed
23842 	72 months ago 	SystemError: ../Objects/longobject.c:998: bad 
argument to internal function 	has patch 		closed
22523 	79 months ago 	[regression] Lib/ssl.py still references 
_ssl.sslwrap 	has patch 		closed

17192   96 months ago   libffi-3.0.13 importhas patch   closed
17579 	97 months ago 	socket module in 2.7.4 raises error instead of 
gaierror in 2.7.3 			closed
14330 	105 months ago 	don't use host python, use host search paths for 
host compiler 	has patch 		closed
4303 	149 months ago 	[2.5 regression] ctypes fails to build on 
arm-linux-gnu 			closed

4469149 months ago  CVE-2008-5031 multiple integer overflows
closed
4552 	150 months ago 	Doc/tools/sphinxext not included in the 2.6.1 
tarball 			closed
4519 	150 months ago 	.pyc files included in 2.6 and 3.0 release 
tarballs 			closed
2601 	157 months ago 	[regression] reading from a urllib2 file 
descriptor happens byte-at-a-time 			closed
984880 	203 months ago 	fix regression on 2.3 branch: Lib/threading 	has 
patch 		closed
598996 	226 months ago 	Failure building the documentation 	has patch 	 
closed


The tradeoffs are the cost of bugfix release candidates (paid most by a 
few people) versus the cost of the relative rare hotfix releases versus 
the frequency of bugfix releases.  I personally prefer a bugfix every 2 
months without candidate to a bugfix every 4 months with one (these 
being alternatives with the same work by the release crew).


A possible compromise would be to cut the release a couple of days ahead 
and then pause while Matthias or anyone else runs tests.  But I would 
understand if Lukasz considered pausing to be worse than finishing the 
release then and there.  Or announce the upcoming release a week ahead 
and ask committers to only merge PRs that could possibly break anything 
to only do so with fresh CI tests.  (Stale tests were one recent reason 
a test-breaking PR got merged.)



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GEXTJNKD7IZL3J4M2ODESLJLLVAEQVKW/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: NOTE: Python 3.9.3 contains an unintentional ABI incompatibility leading to crashes on 32-bit systems

2021-04-03 Thread Terry Reedy

On 4/3/2021 7:15 PM, Miro Hrončok wrote:

On 03. 04. 21 21:44, Łukasz Langa wrote:
The memory layout of PyThreadState was unintentionally changed in the 
recent 3.9.3 bugfix release. This leads to crashes on 32-bit systems 
when importing binary extensions compiled for Python 3.9.0 - 3.9.2. 
This is a regression.


We will be releasing a hotfix 3.9.4 around 24 hours from now to 
address this issue and restore ABI compatibility with C extensions 
built for Python 3.9.0 - 3.9.2.


Thanks for the hotifx.

However, I need to ask: Would this also happen if there was a rc version 
of 3.9.3?


Unless the mistake was just introduced, the mistake would have happened. 
 One this severe would likely have been caught within the week or two 
before a final.  But as Łukasz noted when announcing the change, .rcs 
are generally ignored.  (I suspect that most everyone assumes that 
someone else will test them.  And begging people to not do that does not 
work well enough to justify the release.) 3.8.5 (2020 July 20 was hotfix 
for 3.8.4 (2020 July 14, which did have a candidate, which did not get 
tested the way that 3.8.4 itself was.


--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IN4YTIRP2SE6NGT4GCZXFH6PVNSGU23T/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-02 Thread Terry Reedy

On 4/2/2021 12:02 AM, Guido van Rossum wrote:
On Thu, Apr 1, 2021 at 8:01 PM Terry Reedy 


The current near-Python code does not have such a check.




Again, I'm not sure what "the current near-Python code" refers to. From 
context it seems you are referring to the pseudo code in Mark's PEP 653.



Yes, the part I read was legal Python + $ variables + FAIL.  I should 
have included 'pseudo'.


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JDXXDZ26LFWCGIZKT7XZGRM4CEGXG4TL/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Request for comments on final version of PEP 653 (Precise Semantics for Pattern Matching)

2021-04-01 Thread Terry Reedy

On 4/1/2021 9:38 PM, Guido van Rossum wrote:

On Thu, Apr 1, 2021 at 2:18 PM Mark Shannon > wrote:

Almost all the changes come from requiring __match_args__ to be a tuple
of unique strings.


The current posted PEP does not say 'unique' and I agree with Guido that 
it should not.


Ah, *unique* strings. Not sure I care about that. Explicitly checking 
for that seems extra work,


The current near-Python code does not have such a check.


and I don't see anything semantically suspect in allowing that.


If I understand the current pseudocode correctly, the effect of 's' 
appearing twice in 'C.__match_args__ would be to possibly look up and 
assign C.s to two different names in a case pattern.


I would not be surprised if someone someday tries to do this 
intentionally.  Except for the repeated lookup, it would be similar to a 
= b = C.s.  This might make sense if C.s is mutable.  Or the repeated 
lookups could yield different values.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YAVWTPHGTUDUAOGXDISPKYVD4QMED2HB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Where and how can I contribute?

2021-03-27 Thread Terry Reedy

On 3/27/2021 4:06 AM, Serhiy Storchaka wrote:

26.03.21 20:37, Terry Reedy пише:

On 3/26/2021 6:29 AM, Marco Sulla wrote:

I would contribute to the project in my spare time. Can someone point
me to some easy task? I know C and the Python C API a little.


Review existing PRs.  In some cases (ask), convert existing patches
posted on bpo issues to PRs.


But before doing this ask if the original author want to create a PR.


Asking includes asking any core devs on the issue whether they have 
rejected a particular patch or are interested in possibly merging it 
either as is or with describable changes.


For IDLE issues, I have already asked and gotten approval from most 
authors of pending attached diffs.



--
Terry Jan Reedy


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LTJEEIENLMRQXZG2SPUAP7YGWEP26S6R/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 654: Exception Groups and except* [REPOST]

2021-03-26 Thread Terry Reedy

On 3/26/2021 7:19 PM, Guido van Rossum wrote:

Everyone,

Given the resounding silence I'm inclined to submit this to the Steering 
Council. While I'm technically a co-author, Irit has done almost all the 
work, and she's done a great job. If there are no further issues I'll 
send this SC-wards on Monday.


The current version looks very carefully done.  I leave it to the SC to 
review the details.  But two thoughts on other possible uses.


1. unittests already uses the idea of exception groups by catching 
test_xyz exceptions and adding them to a group, to be printed it when 
all those for a file are run.  If the PEP is accepted, someone might 
request an option to have them packaged as an ExceptionGroup.


2. At least some compilers for other languages can report multiple 
syntax exceptions, but at least for C, the usefulness of more than 2 or 
3 is crippled by the synchronization problem.  Python's compile() only 
reports the first, but I imagine that the usually dependable parsing 
into statements might make multiple reports for Python useful.  I am 
thinking of a 'multiple' mode that repeatedly and recursively did 
'single' compiles. (There would still be only one report per statement 
or compound statement header.)  I have seen newbie code posted on 
Stackoverflow that needed a multi-error report.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CT2SLYRZLNEDCPOSNGFZGTUHRRNJUUBE/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Where and how can I contribute?

2021-03-26 Thread Terry Reedy

On 3/26/2021 6:29 AM, Marco Sulla wrote:

I would contribute to the project in my spare time. Can someone point
me to some easy task? I know C and the Python C API a little.


Review existing PRs.  In some cases (ask), convert existing patches 
posted on bpo issues to PRs.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WVRSZETZC6DE6SEWKTPHR4VZ2GVYOHSK/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: aiter/anext review request

2021-03-19 Thread Terry Reedy

On 3/19/2021 6:11 PM, Joshua Bronson wrote:

Discussion here so far is converging on resurrecting my original PR from 
2018 adding these to operator.

I prefer this too.

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BTHQ5DDLRA4I6UXNCE4ZMUAJULU5LXBS/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Tottime column for cprofile output does not add up

2021-03-10 Thread Terry Reedy

On 3/10/2021 7:16 AM, Jonathan Frawley wrote:


I am using cprofile and PStats to try and figure out where bottlenecks are in a program. 
When I sum up all of the times in the "tottime" column, it only comes to 57% of 
the total runtime. Is this due to rounding of times or some other issue?


pydev list discusses development of future versions, not usage of 
current version.  Please ask your question on python-list.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SKECM7Z5SDZROHA2PLK2SQDJ6OPJL3QL/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Steering Council update for February

2021-03-09 Thread Terry Reedy

On 3/9/2021 3:27 PM, Pablo Galindo Salgado wrote:


The Steering Council just published the community update for February:


Thank you for posting this.


The Steering Council discussed renaming the master branch to main
and the consensus was that we should do that.


I did not that this was an issue.  This will break 4 of my .bat files 
(update repository and workspaces, build python.exes, run sphinx on 
docs,  run coverage.) and, I presume, many other workflow scripts and 
bots.  What is the upside?  ('next (version)' might be more accurately 
descriptive, but 'main' works as the 'main branch we are working on'.) 
Does 'master' confuse people?


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EMXFMHSFF64VCDFPWK7PXHC27CI2J5SM/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-22 Thread Terry Reedy

On 2/22/2021 6:58 AM, Victor Stinner wrote:

On Mon, Feb 22, 2021 at 12:51 PM Ivan Pozdeev via Python-Dev
 wrote:

IIRC I suggested earlier that buildsbots should be integrated into the PR 
workflow in order to make it the contributor's rather than a core
dev's burden to fix any breakages that result from their changes.


The problems is that many 'breakages' do NOT result from the PR changes. 
 For IDLE patches, EVERY CI failure other than in test_idle is a bogus 
failure.  My productivity is already degraded by the current blind 
automated rules.


My point here is that machines literally have no judgment, and that 
automation is not the panacea that people think.  Writing good rules is 
often hard to impossible, and blind application of inadequate rules 
leads to bad results.



Some buildbot worker take 2 to 3 hours per build. Also, it would not
scale. Buildbots are not fast enough to handle the high number of PR
and PR updates.

When there is clear relationship between a buildbot failure and a
merged PR, a comment is added automatically showing the failed test
and explanation how to investigate the issue.


This is false and an example of the above comment.  The notice is sent 
whenever any buildbot test fails after a merge, regardless of whether 
there is any relationship or not. There often (usually?) is not.

 > It's there for 2 years

thanks to Pablo, and so far, I rarely saw developers paying attention
to these failures. They just ignore it.


Every one of those big, bold, ugly notices I have received has been a 
false positive.  I believe they are usually due to one of the flaky 
tests (asyncio or otherwise) that should have been disabled but have not 
been.  Of course people ignore routinely false positives.


If there is an expert for the module whose test file failed, that person 
should be the target of the notice.  If a PR or merge breaks test_idle, 
I would like to know.


Actually, because I am conscientious, and because there was one instance 
years ago where test_idle passed CI and failed on one buildbot, and 
because the notice lists which test file failed (in among the noise), I 
do check that line before deleting the notice.



I'm not trying to blame anyone. Contributing to Python requires a lot
of free time. I'm only trying to explain in length what does the
"maintenance burden" mean in practice, since some people are
pretending that supporting a platform is free.


Blind automated blind rules are also not free.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ICFHS2IUYAYWK6WLFU6XLVXFNDYQCPWC/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Move support of legacy platforms/architectures outside Python

2021-02-22 Thread Terry Reedy

On 2/22/2021 6:20 AM, Victor Stinner wrote:


To have an idea of the existing maintenance burden, look at emails sent to:
https://mail.python.org/archives/list/buildbot-sta...@python.org/

Every single email is basically a problem. There are around 110 emails
over the last 30 years:


30 days, not years.  The 20 visible messages are all have title 
f"Buildbot worker {worker} missing".  All the messages on a day are sent 
at the same time.  Replace with a daily "Builbot workers currently 
missing"?  The individual messages seem useless except possibly if sent 
to individual buildbot owners.

...

Multiple buildbots are "unstable": tests are failing randomly. Again,
each failure means a new email. For example, test_asyncio likes to
fail once every 10 runs (coarse average, I didn't check exactly).


I strongly feel that the individual repeatedly failing tests (only 3 or 
4 I think) should be disabled and the asyncio people notified.  As I 
remember, you once insisted on keeping routinely failing tests.



By the way, these random failures are not only affecting buildbots,
but also CIs run on pull requests. It's *common* that these failures
prevent to merge a pull request and require manual actions to be able
to merge the PR (usually, re-run all CIs).


For backports, it means closing the backport and reopening by re-adding 
a backport label to the master PR.  This is why I really really want the 
repeatedly failing tests disabled.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QGKV7DOVE7XXMGJRAZZNWK7NDPZS4NKJ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Inadequate error reporting during function call setup stage

2021-02-21 Thread Terry Reedy

On 2/21/2021 12:04 PM, Paul Sokolovsky wrote:


Traceback (most recent call last):
   File "pseudoc_tool.py", line 91, in 
 first_class_function_value(func, **pass_params)
TypeError: print() got an unexpected keyword argument 'noann'


This is not typical behavior in current Python (3.8+).

def f(): pass
def g(): f(a=0)
g()

for instance, results in

Traceback (most recent call last):
  File "F:\Python\a\tem3.py", line 3, in 
g()
  File "F:\Python\a\tem3.py", line 2, in g
def g(): f(a=0)
TypeError: f() got an unexpected keyword argument 'a'

def f(): print(b=4)
def g(): f()
g()

gives me

Traceback (most recent call last):
  File "F:\Python\a\tem3.py", line 3, in 
g()
  File "F:\Python\a\tem3.py", line 2, in g
def g(): f()
  File "F:\Python\a\tem3.py", line 1, in f
def f(): print(b=4)
TypeError: 'b' is an invalid keyword argument for print()

Can you create a minimal complete verifiable example and post it on 
bugs.python.org.  This is where bug reports belong, not on pydev and 
python-ideas.



I now implemented that too, and now everything makes sense:

Traceback (most recent call last):
   File "pseudoc_tool.py", line 91, in 
   File "../xforms.py", line 25, in print
TypeError: unexpected keyword argument 'noann'


Since you have a fix in your repository, perhaps you can include a PR 
against cpython master branch.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QK2FWYBY3SJ2KPMJ37HCEF7QM5DDN2MT/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Security releases of CPython

2021-02-19 Thread Terry Reedy

On 2/19/2021 5:11 AM, Michał Górny wrote:

On Thu, 2021-02-11 at 23:24 -0500, Terry Reedy wrote:



Releases are not just a push of a button.  Make the release
job too onerous, and there might not be any more volunteers.


While I understand your concerns and sympathize with them,


Your accusations of unfairness in response to my polite, volunteered 
response, suggest otherwise.



I don't think it's fair to play the volunteer card here.

...

it just feels unfair for you to be dumping the security work on us.


Which I did not do.  Bye.

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/UOWWSWZ3O4NV6PIQBMF2KKBVSTK6HXL3/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Security releases of CPython

2021-02-11 Thread Terry Reedy

On 2/11/2021 3:23 PM, Michał Górny wrote:

Hello,

I'm the primary maintainer of CPython packages in Gentoo. I would like
to discuss possible improvement to the release process in order to
accelerate releasing security fixes to users.

I feel that vulnerability fixes do not make it to end users fast enough.


When we introduce a bad regression in a release, including a 
severe-enough security vulnerability, and discover it soon enough, we 
have sometimes made immediately releases.


Beyond that, your proposal to add several releases per Python version, 
perhaps double what we do now, runs up against the limits of volunteer 
time and effort.  As it is now, becoming a release manager is a 7 year 
commitment, with at least one release about every other month for the 
first 4.  There are also the 2 people who produce the Windows and macOS 
installers.  I believe each has done it for at least 7 or 8 years 
already.  Releases are not just a push of a button.  Make the release 
job too onerous, and there might not be any more volunteers.



For example, according to the current release schedules for 3.9 and 3.8,
the bugfix releases are planned two months apart. While the release is
expected to happen in the next few days, both versions are known to be
vulnerable for almost a month!

Ironically, things look even worse for security-supported versions.
Please correct me if I'm wrong but both 3.7 and 3.6 seem to be behind
schedule (planned for Jan 15th), and they are known to be vulnerable
since mid-October.

In my opinion, this causes three issues:

1. Users using official releases are exposed to security vulnerabilities
for prolonged periods of time.

2. When releases happen, security fixes are often combined with many
other changes. This causes problems for distribution maintainers who, on
one hand, would like to deploy the security fixes to production versions
ASAP, and on the other, would prefer that the new version remained in
testing for some time due to the other changes.

3. Effectively, it means that distribution developers need to track
and backport security fixes themselves. In the end, this means a lot of
duplicate work.


Perhaps people doing duplicate work could get together to eliminate some of
the duplication.  It should be possible to filter security commits from the
python-checkins list by looking at the news entries and if need be, the bpo
issues.


I think that security fixes are important enough to justify not sticking
to a strict release schedule. Therefore, I would like to propose that if
vulnerability fixes are committed, new releases are made
as frequently as necessary and as soon as possible (i.e. possibly giving
some time for testing) rather than according to a strict schedule.

Furthermore, I think that at least for branches that are in higher level
of maintenance than security, it could make sense to actually make
security releases (e.g. 3.9.1.x) that would include only security fixes
without other changes.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4S5LOTVJZUA5PQ5TRGQFCWHTGA4BOXBO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 597: Add optional EncodingWarning

2021-02-09 Thread Terry Reedy

On 2/9/2021 8:28 PM, Inada Naoki wrote:


Note that many Python users don't use consoles.


Those of use who do may find it hard to imagine just how easy we have 
made computing.


My daughter minored in Computer Science about 6 years ago.  She never 
saw Command Prompt until the summer after her Junior year when she 
watched me use it to install numpy and other packages for her.  I had to 
do it because 'Run pip install numpy', etc, was met with a blank stare. 
 I had taught her Python with IDLE, downloaded and install with a 
browser, and had neglected to teach her 'Dos' until then.


So had her CS classes.  Those previous used Racket in a Dr. something 
environment and Java in, I believe, Eclipse.  Also downloaded and 
installed with a browser.



They just starts
Jupyter Notebook, or they just write .py file and run it in the
Minecraft mods.


Also similar.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O5SS7I2H2GRCM2YBOVYX7CWUYTX2J7AO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 647 (type guards) -- final call for comments

2021-02-09 Thread Terry Reedy

On 2/9/2021 11:21 AM, Guido van Rossum wrote:
I think we have reached consensus on PEP 647 in typing-sig. We have 
implementations for mypy and pyright, not sure about the rest. This PEP 
does not affect CPython directly except for the addition of one special 
item (TypeGuard) to typing.py -- it would be nice to get that in the 
3.10 stdlib.


I'm CC'ing python-dev here to see if there are any further comments;


As a naive possible future user of typing, I have a couple of related 
comments on the presentation and suggested additions with regard to the 
type of TypeGuard 'instances' and the meaning of type guard function 
returns.


"Its return type would be changed from bool to TypeGuard[List[str]]."

My first thought was "Huh? The return type *is* a bool!"  Then I 
realized that a TypeGuard[List[str]] is a 'specified affirmative bool', 
a bool for which True means 'is a List[str]'.  More generally, a 
TypeGuard[X] is a bool for which True affirmatively means that the first 
argument of the function is specifically an X.  I mentally condensed 
that to 'bool subtype' and suspect that others have and will do the 
same, but you later reject 'subtype' because 'subtype' has a specific 
meaning in the typing world that does not apply here.


Suggestion 1:  Rather than mere say what a TypeGuard is not, follow the 
quoted sentence with a statement of what it is, such as I gave above.


"Specification\n TypeGuard Type \n "

The first paragraph specifies that the function must be a predicate 
(returns bool and only bool).  It leaves out that it must be a positive 
or affirmative predicate for which 'True' says what the first argument 
is, rather than what it is not.  The examples meet this requirement and 
narrowing only for 'if' clauses implies this requirement, but ...


Suggestion 2: End the first paragraph by stating the requirement.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/MF2DRQ3GIBNGFOUDWL4ZSVPBCQPVYVQO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Change windows installation program name

2021-02-08 Thread Terry Reedy

On 2/8/2021 3:12 PM, Ivan Pozdeev via Python-Dev wrote:

You want to make a poll or something?

Discourse can do that: 
https://meta.discourse.org/t/how-to-create-polls/77548


On 08.02.2021 23:07, Barry Scott wrote:

I raise https://bugs.python.org/issue43156 that suggests that
new users on windows would be less confused if the name of the
installation program did not seem to imply it was the python program.

In a nutshell the proposal is to change from this:

   python-3.10.0a5.exe
   python-3.10.0a5-amd64.exe

to this (thanks to Matthew Barnett for the improved names:

   python-3.10.0a5-win32-setup.exe
   python-3.10.0a5-win64-setup.exe


No poll needed, just agreement from the Window people and in particular 
the person (Steve Dower) who makes the file.  Marking the components 
'installation' and 'Windows', which I just die, will get their/his 
attention.  (As author, you can also mark components.)


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/Q4T652EFSTX4R4GD7RSZLDQTHQIKPNCL/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PR Review request - bpo-41928: Add support for Unicode Path Extra Field in ZipFile

2021-02-08 Thread Terry Reedy

On 2/7/2021 1:55 PM, Andrea Giudiceandrea via Python-Dev wrote:

Hi Python Dev team,
I submitted a PR https://github.com/python/cpython/pull/23736 two months ago.

The PR, which fixes an issue https://bugs.python.org/issue41928 in ZipFile, is 
"awaiting core review".


I nosied and requested a review from the active zipfile 'expert' (Serhiy).


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RWZDYKBRQRTXYUCC6RSA6LF77YJ32YCO/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: What's up with assignment expression and tuples?

2021-02-05 Thread Terry Reedy

On 2/5/2021 2:51 AM, Paul Sokolovsky wrote:


https://www.python.org/dev/peps/pep-0572/

...


((a, b) := (1, 2))

   File "", line 1
SyntaxError: cannot use assignment expressions with tuple


Why this accidental syntactic gap?


As should be clear from reading "Differences between assignment 
expressions and assignment statements", this 'gap' in entirely 
intentional, not accidental.  *All* elaborations of 'name := expression' 
are listed and rejected as outside the scope of the proposal, which was 
to keep one reference to the expression value for later use.  At least 
some of these elaborations were suggested and rejected during the 
voluminous discussion.


The principal a.e. use in conditional expressions is testing for 
non-nullness.  Your


> while ((a1, b1) := phi([a0, a2], [b0, b2]))[0] < 5:
>a2 = a1 + 1
>b2 = b1 + 1

is an unusual and very specific use.  You want to have your tuple (for 
subscripting for testing) and eat it too (by unpacking).  One can 
instead separate unpacking from testing a couple of ways


while (tup := phi([a0, a2], [b0, b2]))[0] < 5:
a2, b2 = tup
a2 = a1 + 1
b2 = b1 + 1


while True:
a1, b1 = phi([a0, a2], [b0, b2])
if a1 >= 5: break
a2 = a1 + 1
b2 = b1 + 1


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/VM3OJ4E73DQXHNUB2JV67JALI2EEO3DP/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 563: get_type_hints should use closure when available

2021-02-03 Thread Terry Reedy

On 2/2/2021 8:37 PM, Caleb Donovick wrote:
The discussion around PEP 649 got me thinking about what I believe is 
the largest downside to PEP 563: the inability to evaluate annotations 
created with closures.  While this is in general unavoidable,  if 
the type is ever referenced in an annotated function (including as an 
annotation) it should be resolvable via `__closure__`.


For example:
```
from __future__ import annotations
import typing
def gen(T):
     def f(x: T):
         y: T = ...
     return f

f = gen(int)
nonlocal_vars = {
   var : cell.cell_contents
   for var, cell in zip(f.__code__.co_freevars, f.__closure__)
}
assert typing.get_type_hints(f, localns=nonlocal_vars)  == {'x': int}
```

I would just open a PR to have `get_type_hints` attempt to resolve 
closure variables by default.  However, this would require an update to 
PEP 563 and I don't know what the protocol is there.


Once a PEP is accepted and implemented, the PEP document itself is 
frozen and not updated.  The Python language and docs and the CPython 
implementation are still subject to what is hopefully further progress.


You are free to open an thread on python-ideas list or an issue on bpo, 
optionally with a PR, or both.  It would be good to reference the PEP 
and note whether this idea appears there or not.  Further discussion of 
the idea itself would depend on whether is was a rejected alternative, a 
deferred addition, or just mentioned.


A speculative PR for an idea not yet accepted in principle may be 
rejected but may help gain acceptance.


PS: I don't use type hints yet and have not much opinion about about 
your idea (it naively seems plausible), so the above is a generic answer.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FQLVHTIMDEI6IPO5FFL2ZIUKZC5RQK4U/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: What is the pyclbr public API?

2021-01-29 Thread Terry Reedy

On 1/29/2021 8:57 PM, Guido van Rossum wrote:
On Fri, Jan 29, 2021 at 5:39 PM Terry Reedy <mailto:tjre...@udel.edu>> wrote:


Guido, thank you for the helpful discussion.  I now think that we
should
just add 'end_lineno=None' at the end of the Function/Class __init__
signatures.  pyclbr makes one call to each to contruct the tree it
returns, and the test functions make another call to each.

If those are the only two calls to each, it hardly matters what they
look like.  If there are non-stdlib calls, it is not worth breaking
them. And others might well localize their Function/Class calls within
wrappers similar to _nest_function() and _nest_class().


Okay, I wasn't quite ready to recommend this, but I agree that that's 
the most compatible solution. (Put a `*` before the optional arg so it 
must be specified as a keyword.)


I presume you mean immediately before 'end_lineno', as putting it before 
all optional params would break positional passing.



A more important pyclbr issue, I think, is that readline and
readline_ex
return a 'half node', a dict of children, instead of a Module node.  It
is a nuisance, such as when constructing IDLE's module browser
tree.  It
is the same as if ast_parse were to return the body list of ast.Module
instead of ast.Module itself.  A readmodule() function could return a
proper tree with a root Module node, with attributes file, name, lineno
(1), end_lineno, and children.

Sounds good. Thanks for caring about this very old module! (I am not 
100% sure of its origins but I think it may have been written in 
Python's earliest years by one of my office mates, Sjoerd Mullender.)


Adding a new API would be an opportunity to 'do it right'.  The test of 
it being an improvement is if it allows simplification of the module 
browser code.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KAQBTNC5R5IHHXKPUIVMKAPFTNIHE7BK/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: What is the pyclbr public API?

2021-01-29 Thread Terry Reedy
Guido, thank you for the helpful discussion.  I now think that we should 
just add 'end_lineno=None' at the end of the Function/Class __init__ 
signatures.  pyclbr makes one call to each to contruct the tree it 
returns, and the test functions make another call to each.


If those are the only two calls to each, it hardly matters what they 
look like.  If there are non-stdlib calls, it is not worth breaking 
them. And others might well localize their Function/Class calls within 
wrappers similar to _nest_function() and _nest_class().


A more important pyclbr issue, I think, is that readline and readline_ex 
return a 'half node', a dict of children, instead of a Module node.  It 
is a nuisance, such as when constructing IDLE's module browser tree.  It 
is the same as if ast_parse were to return the body list of ast.Module 
instead of ast.Module itself.  A readmodule() function could return a 
proper tree with a root Module node, with attributes file, name, lineno 
(1), end_lineno, and children.



On 1/28/2021 11:32 PM, Guido van Rossum wrote:
On Thu, Jan 28, 2021 at 8:08 PM Terry Reedy <mailto:tjre...@udel.edu>> wrote:


On 1/27/2021 7:01 PM, Guido van Rossum wrote:
 > It's likely that the additions are going to break someone's code;

Since at least 2007, when Georg moved the 3i reST tree into the 3k
branch, the instances have been described as "Class/Function Descriptor
Objects", returned by readmodule(_ex), that "provide the following data
members".  There are no methods (maybe __repr__ should be added for
debugging), so these are pure data classes, identical in purpose to
pure
dataclasses.


Lots of people just read the source code though. In general we've often 
been careful with changing undocumented APIs if we believe they are 
commonly used in ways not endorsed by the official documentation.


The only intended reason to call these would be to create a tree for
testing. 



Really? Couldn't someone have an application where they insert new 
objects (like is documented for the ast module)?


We do this indirectly with a couple of private test-only
pyclbr functions.  Being private, *those* functions can have the new
parameter added anywhere, regardless of the Class/Function signatures.

test_pyclbr uses a constructed tree as the expected output from
readline_ex(test_code).  IDLE's test_browser uses a construction as
test
input to the browser widget. If someone does similar testing, it could
break with the proposed changes.


So the question is whether we care. I don't know how to assess that.

 > the
 > question seems to be do we care, since the constructors are
undocumented?

If any library functions return dataclasses, are the signatures of the
(possibly generated) __init__ functions documented?


I'd say so, because we document how `@dataclass` constructs the 
`__init__()` method.


I care more about breaking tests than possibly non-existent 'off-label'
uses, but Python won't know.

 > But shouldn't we just document the signatures,

The existing or proposed signatures?


I meant the proposed signatures.

 > so that in the future
 > we're not going to make the same mistake?

I am not sure what you mean by the 'mistake' to not be repeated.  I
consider it be had been either not declaring the signature private
or at
least not making optional parameters keyword only.


Yeah, by 'mistake' I meant leaving it ambiguous whether the constructor 
signature is part of the API. And yes, we could also have designed the 
constructor signatures more carefully.


 > I also wonder if some judicious use of keyword-only args might help
 > users who currently call the old constructors catch the breakage
where
 > it's easily diagnosed (i.e. at the call site) instead of once the
object
 > is instantiated (e.g. "why is end_lineno/parent the wrong type")?

With 'end_lineno' added after existing required parameters, just before
the existing optional parameter 'parent', an existing Class call ending
with '..., posint, parent=SomeClass)' would fail with

TypeError: Class.__init__ missing 1 required positional argument:
'end_lineno'


Unless you give end_lineno a default (I don't think we should do this 
unless we find significant rogue usages).


If parent had previously been made 'keyword only', a call ending
instead
with '..., num, SomeClass' would be a failure now.  But since not,


Right. One of the mistakes.

 > Perhaps even add some type checks to catch obvious mistakes
(could be as
 > simple as `assert isinstance(end_lineno, int)").

Do you consider this enough to make the proposed positioning of
'end_lineno' acceptible?  The tradeoff is some immediate annoyance
versus forever annoyance of a misplaced required 'optiona

[Python-Dev] Re: What is the pyclbr public API?

2021-01-28 Thread Terry Reedy

On 1/27/2021 7:01 PM, Guido van Rossum wrote:

It's likely that the additions are going to break someone's code;


Since at least 2007, when Georg moved the 3i reST tree into the 3k 
branch, the instances have been described as "Class/Function Descriptor 
Objects", returned by readmodule(_ex), that "provide the following data 
members".  There are no methods (maybe __repr__ should be added for 
debugging), so these are pure data classes, identical in purpose to pure 
dataclasses.


The only intended reason to call these would be to create a tree for 
testing.  We do this indirectly with a couple of private test-only 
pyclbr functions.  Being private, *those* functions can have the new 
parameter added anywhere, regardless of the Class/Function signatures.


test_pyclbr uses a constructed tree as the expected output from 
readline_ex(test_code).  IDLE's test_browser uses a construction as test 
input to the browser widget. If someone does similar testing, it could 
break with the proposed changes.


the 
question seems to be do we care, since the constructors are undocumented?


If any library functions return dataclasses, are the signatures of the 
(possibly generated) __init__ functions documented?


I care more about breaking tests than possibly non-existent 'off-label' 
uses, but Python won't know.



But shouldn't we just document the signatures,


The existing or proposed signatures?

so that in the future 
we're not going to make the same mistake?


I am not sure what you mean by the 'mistake' to not be repeated.  I 
consider it be had been either not declaring the signature private or at 
least not making optional parameters keyword only.


I also wonder if some judicious use of keyword-only args might help 
users who currently call the old constructors catch the breakage where 
it's easily diagnosed (i.e. at the call site) instead of once the object 
is instantiated (e.g. "why is end_lineno/parent the wrong type")?


With 'end_lineno' added after existing required parameters, just before 
the existing optional parameter 'parent', an existing Class call ending 
with '..., posint, parent=SomeClass)' would fail with


TypeError: Class.__init__ missing 1 required positional argument: 
'end_lineno'


If parent had previously been made 'keyword only', a call ending instead 
with '..., num, SomeClass' would be a failure now.  But since not,


Perhaps even add some type checks to catch obvious mistakes (could be as 
simple as `assert isinstance(end_lineno, int)").


Do you consider this enough to make the proposed positioning of 
'end_lineno' acceptible?  The tradeoff is some immediate annoyance 
versus forever annoyance of a misplaced required 'optional' parameter.


Since I have in mind a possible IDLE use for end_lineno, quite different 
from that of the patch author, I consider either way of adding it 
superior to never adding it.



On Wed, Jan 27, 2021 at 2:36 PM Terry Reedy <mailto:tjre...@udel.edu>> wrote:


pyclbr is the stdlib module browser (once just class browser, hence the
name).  The doc
https://docs.python.org/3/library/pyclbr.html#module-pyclbr
<https://docs.python.org/3/library/pyclbr.html#module-pyclbr>, which I
revised in 2017, documents readline and readline_ex as the public call
interface.  The functions return a hierarchical structure that includes
Function and Class instances.  (Both subclass pyclbr._Object.)  The doc
I revised already omitted signatures for these classes and just listed
the attributes of instances.  Just before that listing, I added "Users
are not expected to create instances of these classes."

https://bugs.python.org/issue38307
<https://bugs.python.org/issue38307> and
https://github.com/python/cpython/pull/24348
<https://github.com/python/cpython/pull/24348>
propose to add an end_lineno attribute. Since pyclbr is now, since last
November, based on the ast tree and since the relevant ast nodes
have an
end_line attribute, the proposal amounts to copying that attribute,
along with others, instead of ignoring it.

The patch proposes to add 'end_lineno' after existing start 'lineno' in
the _Object, Function, and Class signatures.  I prefer doing this,
rather than adding the new parameter at the end of the list, because a)
being the the logical place would make the code easier to read, and b)
new names as the end of the signature, follow optional arguments, would
have to be optional, whereas

My question is whether the omission of signatures and my added sentence
sufficiently defines the signatures of these classes (in particular,
the
argument order) as private to approve the patch as is?


-- 
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
<mailto:python-dev@python.org>
To unsu

[Python-Dev] Re: Why aren't we allowing the use of C11?

2021-01-28 Thread Terry Reedy



On 1/28/2021 11:26 AM, Mark Shannon wrote:


PEP 7 says that C code should conform to C89 with a subset of C99 allowed.


As I remember, when the Python C dialect was discussed about 4 years 
ago, no one seriously proposed moving beyond a subset of C99 addition, 
because of the state of MSVC.


It's 2021 and all the major compilers support C11 (ignoring the optional 
parts).


C11 has support for thread locals, static asserts, and anonymous structs 
and unions. All useful features.


Is there a good reason not to start using C11 now?


Inertia is *a* reason.  C11 would require (and allow!) most (nearly 
all?) people compiling CPython on Windows to upgrade their VS-VC setup. 
 Maybe including some buildbot machines.


Judging from the post for
https://bugs.python.org/issue42380
this requires VS 16.8 with MSVC 14.28, released last November.  Prior 
distributions of VS 2019 had earlier versions.  Last I knew, regular 
people without a dev license cannot get just MSVC.  So we cannot require 
a compiler most people cannot get.  This post otherwise gives additional 
reasons why upgrading would be good.


https://bugs.python.org/issue39952 discusses issues compiling with VS 
2019 last spring (compiler not specified.  The main blocker was tcl/tk.
3.9 uses 8.6.9; 3.10 is using 8.6.10 and I hope this is upgraded to 
3.6.11 before release.


The devguide needs updating, as it says to get VS 2017, while 
https://visualstudio.microsoft.com/ only distributes VS 2019.  It 
appears that C11 would require VS 2019 16.8 or later.


https://en.wikipedia.org/wiki/Microsoft_Visual_C%2B%2B#Internal_version_numbering 
has a table of MSVC versions, _MSC_VERs, and VS 20xx releases.  Each of 
'VS 2017' and 'VS 2019' are not just one thing but comprise multiple 
releases/versions, just like 'Windows 10'.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/PAWQQIGS2HJQ5F7GD5BF2YPOI5EAPKTG/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] What is the pyclbr public API?

2021-01-27 Thread Terry Reedy
pyclbr is the stdlib module browser (once just class browser, hence the 
name).  The doc
https://docs.python.org/3/library/pyclbr.html#module-pyclbr, which I 
revised in 2017, documents readline and readline_ex as the public call 
interface.  The functions return a hierarchical structure that includes 
Function and Class instances.  (Both subclass pyclbr._Object.)  The doc 
I revised already omitted signatures for these classes and just listed 
the attributes of instances.  Just before that listing, I added "Users 
are not expected to create instances of these classes."


https://bugs.python.org/issue38307 and
https://github.com/python/cpython/pull/24348
propose to add an end_lineno attribute. Since pyclbr is now, since last 
November, based on the ast tree and since the relevant ast nodes have an 
end_line attribute, the proposal amounts to copying that attribute, 
along with others, instead of ignoring it.


The patch proposes to add 'end_lineno' after existing start 'lineno' in 
the _Object, Function, and Class signatures.  I prefer doing this, 
rather than adding the new parameter at the end of the list, because a) 
being the the logical place would make the code easier to read, and b) 
new names as the end of the signature, follow optional arguments, would 
have to be optional, whereas


My question is whether the omission of signatures and my added sentence 
sufficiently defines the signatures of these classes (in particular, the 
argument order) as private to approve the patch as is?



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SQWNNGY7WADHFGAIQZRIMPPLYJGIV4TZ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Speeding up CPython

2021-01-24 Thread Terry Reedy

On 1/24/2021 6:09 PM, Bruno Cabral wrote:

Hello Everyone,

I´m very curious about this proposal, but unfortunately it has been a 
while since I heard any news about this project.

Does anyone know what happened?


The Python Software Foundation currently has a shortfall of funds rather 
than a surplus.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ILJPVZF3F2ERITJ3FAWBORY2A7VKBYGX/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 651 -- Robust Overflow Handling

2021-01-19 Thread Terry Reedy

On 1/19/2021 10:01 AM, Mark Shannon wrote:


The following program will run safely to completion:


I interpreted this to mean 'works now', on whatever system you tested 
this on.  You question suggests that you meant "fails now but will work 
with a successful patch for the PEP".



 sys.setrecursionlimit(1_000_000)


On Windows, this recursion limit increase lets the function run about 
2160 loops (when run with IDLE) instead of 1000, but makes the behavior 
WORSE by replacing the exception with a silent failure.  This could be 
considered worse than a crash.



 def f(n):
 if n:
 f(n-1)

 f(500_000)


I'm not sure whether you are saying that this doesn't work now, that it 
can't work, or that it shouldn't work.



If that it doesn't work now, then I agree.


On my Win 10, setting the recursion limit to 2160 (2135 in IDLE, which 
bumps it up to account for additional stack space used by IDLE) results 
in RecursionError.  Much higher and the failure is silent.  Always 
getting a RecursionError would itself be an improvement.


So I am saying that current recursion limit handling on Windows is 
terrible and perhaps worse than on Linux.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/XSPGPFHIP5VWDBEWQHKSU4A55DHFWAGF/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 651 -- Robust Overflow Handling

2021-01-19 Thread Terry Reedy

On 1/19/2021 8:31 AM, Mark Shannon wrote:


It's time for yet another PEP :)

Fortunately, this one is a small one that doesn't change much.
It's aim is to make the VM more robust.

Abstract


This PEP proposes that machine stack overflow is treated differently 
from runaway recursion.


My impression is that this is already the case in the sense given below.

https://bugs.python.org/issue42887 is about segfaults with this code 
with no Python-level recursion.


mystr  = "hello123"
for x in range(100):
mystr = mystr.__sizeof__()

Christian Heimes reproduced and said "The stack trace is several hundred 
thousand (!) levels deep."
Ronald Oussoren said "The dealloc of "mystr" will cause recursive calls 
to tp_dealloc along the entire chain and that can exhaust the C stack."


Since hundreds of thousands is a lot bigger than 1000, I assume that the 
C recursion is not tracked.  Whick is to say, recursive C calls are not 
counted, unlike recursive Python calls.  There have been several issues 
about highly nested code and recursive structures causes segfaults, 
presumably due to C level recursion.  These are usually closed as 'Won't 
fix' and the suggestion 'don't do that'.


I strongly agree that a fix would be great.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4NA42RO7E5XV75WHWBEFVSJMFHJ7FCUJ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 651 -- Robust Overflow Handling

2021-01-19 Thread Terry Reedy

On 1/19/2021 8:31 AM, Mark Shannon wrote:

Hi everyone,

It's time for yet another PEP :)

Fortunately, this one is a small one that doesn't change much.
It's aim is to make the VM more robust.

Abstract


This PEP proposes that machine stack overflow is treated differently 
from runaway recursion. This would allow programs to set the maximum 
recursion depth to fit their needs and provide additional safety 
guarantees.


The following program will run safely to completion:

     sys.setrecursionlimit(1_000_000)

     def f(n):
     if n:
     f(n-1)

     f(500_000)


Are you sure?  On Windows, after adding the import
and a line at the top of f
if not n % 1000: print(n)
I get with Command Prompt

C:\Users\Terry>py -m a.tem4
50
499000
498000

C:\Users\Terry>

with a pause of after 1 to multiple seconds.  Clearly did not run to 
completion, but no exception or Windows crash box to indicate such 
without the print.


In IDLE, I get nearly the same:
= RESTART: F:\Python\a\tem4.py
50
499000
498000

 RESTART: Shell
>>>
The Shell restart indicates that the user code subprocess crashed and 
was restarted.  I checked that sys.getrecursionlimit() really returns 
1_000_000.


To show completion, do something like add global m and m+=1 in f and m=0 
and print(m) after the f call.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KNWNOKK3QJVS3DCHNY3FOFSVMVU3EVMA/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: seg fault in 3.10a4

2021-01-17 Thread Terry Reedy

On 1/17/2021 8:05 AM, Stestagg wrote:

Hi Robin

It would be ideal if you could please create a new issue here: 
https://bugs.python.org/ 


If 'reportlab userguide creation' uses any 3rd party compiled C code, 
this may be premature.  bugs.python.org is for patching cpython.  Seg 
faults with 3rd party extensions are often, even usually due to a bug in 
the extension that we cannot fix, or for new python version, a missing 
upgrade.


The questions Robin asked seem like a reasonable start.  He might get 
additional suggestions from python-list from someone who has seen 
something similar.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6SRTWMOHBGKPXTTUBSU477YEPOZSJR46/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-13 Thread Terry Reedy

On 1/13/2021 8:56 PM, Greg Ewing wrote:

On 14/01/21 1:13 pm, Paul Sokolovsky wrote:

But nobody talked about optimizing away generic "pure"-annotated
functions (which would differ from "mathematical" definition of
purity), only about optimizing "pure" *dunder* methods


The same thing applies. If we decide that print() is pure,
then a __bool__ that calls print() is also pure, so there's
nothing wrong with optimising it away, right?


I say 'yes', because the purpose of logging is to document what happens, 
and if nothing happens, there is nothing to document.  Wrapping a 
.__bool__ in a logging decorator might be part of testing it.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/NRJT2YZ2U73ZYFZPAKKPXFBKGLJMBG3M/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: How to add multiple python kernel(2.7.x,3.6.x,3.7.x) to jupyterhub

2020-12-20 Thread Terry Reedy

On 12/20/2020 12:24 PM, Shaik Zainul wrote:

Hi,
i am new to jupyterhub and i am trying to setup
1. add multiple python kernel(2.7.x,3.6.x,3.7.x) to jupyterhub
2. setup user specific kernel - means i have 3 kernels PY3, Pyspark and 
R. how can i assign this 3 PY3, Pyspark and R kernels to User1, User2 
and User3 respectively on Jupyterhub.

[snip help request]

pydev list is for development *of* future Python versions and CPython 
releases.  Ask elsewhere for help on *using* Python, and especially 3rd 
party tools that we did not develop.  There must be a list for IPython 
and Jupyter.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CHHMDNEK44TJNXL5AYKIITI7ORL7EW3C/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: NEWS, changelog, and blurb -- a teaching moment

2020-12-09 Thread Terry Reedy

On 12/9/2020 2:19 PM, Ethan Furman wrote:

Greetings!

I'm hoping somebody can alleviate my confusion.  I had thought that the 
blurb tool was created to resolve the near-constant push races when the 
NEWS file was updated; but it appears that it only affects the change log.


Unreleased blurbs are put in Misc/NEWS.d/next.

For each release of a branch, the blurb are removed from .../next and 
consolidated into Misc/NEWS.d/x.y.z??, such as 3.10.0a3.  PRs changing 
these file should only correct errors and not any other files.  There 
should also never be push races.


If I have a change that really needs to be in the NEWS file itself, do I 
need to directly edit that file?


By NEWS, do you mean, for instance, Doc/whatsnew/3.10.rst?  If you want 
to make sure that a new feature you merge is included, yes, add it to 
the file,  Because of the fewer number of entries and much finer 
categorization, and hence more random insert locations, conflicts happen 
much less often than with the old news logs.  Also, contributors do not 
usually add a What's New entry unless and until asked, and that can be 
delayed until a PR is sure to be merged somewhat soon.


Anything you write may be edited by the release manager or other editor. 
 If the devguide says anything about this file, I could not find it. 
The devguide index only indexes PEP references.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GH47YTRTT7GAXNFSY4D4MQRUWGRZJW7Q/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: The semantics of pattern matching for Python

2020-11-20 Thread Terry Reedy
Mark, did you get the response I sent to hotpy.org 4 days ago?  Is that 
a real address?  I ask because the typos I reported are still there and 
trying to visit hotpy.org fails.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/TX7W5TVIJT5BAMP3ZEIS6QSOVJNHFDQA/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Review patch fixing packed bitfields in ctypes struct/union

2020-11-20 Thread Terry Reedy

On 11/20/2020 4:03 AM, Simon Cross wrote:


Thank you for this [cttpes] patch! > I can't help land it, but it looks sane to 
me.


If you have a github account, you can help by reviewing it.  Check the 
spelling, grammar, and clarity of comments, docstrings, and news item. 
Can the code be improved.  If you have a local cpython clone, make a 
branch from the PR and test it.  Does it really fix the issue?



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BTPRQXSISHLLA6KXGASQTL3XPVTHUEWU/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Questions about about the DLS 2020

2020-11-16 Thread Terry Reedy

On 11/16/2020 11:57 AM, Tobias Kohn wrote:

1.  This really comes down to how you look at it, or how you define 
pattern matching.  The issue here is that the concept of pattern 
matching has grown into a large and somewhat diverse flock of 
interpretations and implementations (as a side note: interestingly 
enough, some of the only universally agreed-upon standards are to use 
`_` as a wildcard and not to mark names that capture/bind values---which 
are quite exactly the points most fiercely debatted here).


_ looks OK to me.  I was not aware of the 'standard' for name usage. 
But I just noticed the following:


Anyway, the paper presents the pattern matching structure we are 
proposing as one of three major variants of pattern matching:

(a)  Matching arguments to parameters in a function call,


Binding names are in the function definition and parameter signature, 
which is the pattern.  Except for default argument expressions, value 
names are in the call that provided the data to be matched.



(b)  Matching elements to elements in iterable unpacking,


Binding names are in the target pattern on the left.  Value names are in 
the data source expression on the right.


(c)  Matching tree-like data to general patterns in a conditional 
pattern matching structure.


Oh dear.  We need both binding and value names in patterns.  In
  case BinOp(Num(left), '+', Num(right)):
obviously callable names BinOp and Num are treated as value names, with 
their signatures providing subpatterns.  The not-obviously-callable 
names left and right are treated as binding names.


I think that Python matching should use the all-caps convention.  I like 
reusing the existing 'as x' syntax.


I see if-elif-else structures, with assignment expressions included, as 
existing but limited conditional pattern matching structures.  Binding 
names are clearly marked with ':='.

---

Separate item: The Program 1 fact def needs a guard "if n > 1 and int(n) 
== n" to not raise Recursion error for negative and fractional args.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/LGGQ5GDOJMCQGA77XSRXVLEZSHPSOWV4/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Questions about about the DLS 2020

2020-11-16 Thread Terry Reedy

On 11/16/2020 6:14 AM, Mark Shannon wrote:

2. Is the error in the ast matching example, an intentional 
"simplification" or just an oversight?


The example:

```
def simplify(node):
     match node:
     case BinOp(Num(left), '+', Num(right)):
     return Num(left + right)
     case BinOp(left, '+' | '-', Num(0)):
     return simplify(left)
     case UnaryOp('-', UnaryOp('-', item)):
     return simplify(item)
     case _:
     return node

```

is wrong.


This is not 'wrong' because the paper did not specify the signatures or 
definition of BinOp and UnaryOp or a specific module/package of origin. 
 I took these as hypothetical classes in a hypothetical module that 
happens to use what I presume are ast class names.




The correct version is

```
def simplify(node):
     match node:
     case BinOp(Num(left), Add(), Num(right)):
     return Num(left + right)
     case BinOp(left, Add() | Sub(), Num(0)):
     return simplify(left)
     case UnaryOp(USub(), UnaryOp(USub(), item)):
     return simplify(item)
     case _:
     return node
```


Without knowing the signatures of the ast classes, having not worked 
with them as you have, passing instances of an Add class instead of an 
'add' function looks wrong.  How is one Add() different from another? 
For the didactic purpose of the example, this looks worse to me. For me, 
it is harder to read.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/HH26LZOEN446UWKP3J3OFWZ7AYMV2F5A/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Please do not remove random bits of information from the tutorial

2020-11-08 Thread Terry Reedy

On 11/8/2020 5:51 PM, Inada Naoki wrote:

On Mon, Nov 9, 2020 at 3:46 AM Riccardo Polignieri via Python-Dev
 wrote:



Rather, I am slightly concerned about the method in itself - that a deletion
may occur following only a brief exchange on the bug tracker.



Again, I did it following not only a brief exchange on the b.p.o., but
also long discussion in Python-dev.


When I act on an issue on the basis of a pydev discussion, I hopefully 
remember to avoid such after-the-fact concerns by mentioning the thread 
with a title and brief summary.  Example: "In pydev thread '', 3 of 
4 coredevs agreed with this change." (Made up numbers do not refer to 
any particular issue.)  FWIW, I agreed with this particular change also.


> exception chaining is described well in other place

If on the ball, I would have mentioned this also, and where.

--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SEVFT4NXFQ3547HZVYZRRBEWNR4TPONL/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Drop Solaris, OpenSolaris, Illumos and OpenIndiana support in Python

2020-10-31 Thread Terry Reedy

On 10/30/2020 5:08 PM, Garrett D'Amore via Python-Dev wrote:

I’m not on this list.  But I have offered to help - if there are tasks that 
need to be done to help this I can help put the weight of a commercial entity 
behind it whether that involves assigning our developers to work on this, 
helping pay for external developers to do so, or assisting with access to 
machine resources.



What’s more likely is that some group of developers aren’t interested in 
supporting stuff they don’t actively use.

[snip]
I think it is more of a matter of pride in putting out a quality product 
and not making claims that might be false. Anyway, here is what I posted 
on the issue.

https://bugs.python.org/issue42173
"""
Dear Solaris Python fans (about 20 so far here): Here is the situation. 
There are 1000s of open issues on this tracker and at least 100s of open 
cpython PRs, and only 20-30 core developers, mostly volunteers, actively 
(depending on definition) merging PRs and maybe another 10 triagers 
helping to manage issues.


You all can help on issues by checking whether reported bugs still exist 
with current Python (3.8+) on current 'Solaris' (which includes what?). 
Searching open issues for 'Solaris' in 'All text' just returned 114 
hits.  About half were last touched over two years ago, and sometimes 
the last touch was inconsequential (a version or nosy change).  I 
suspect many of the 114 are obsolete.  For example, the last comment, 
five years ago, on #1471934, says the problem was fixed in Solaris 11.2. 
 Does this mean the issue should be closed?


The bottleneck for merging PRs is reviewing PRs.  We coredevs cannot do 
enough reviews ourselves.  But any competent user can  help.  Reviewing 
has two components.  First, does the patch fix the problem?  Testing 
this requires a Github account and a local clone and ability to build a 
test binary.  See devguide.python.org for more.  Second, does the patch 
meet our standards of code quality.  Solaris-specific patches likely 
change the C part of cpython, so C competence and understanding of PEP 7 
is needed here.

"""

Of the 114 issues mentioned above, not all are Solaris specific, and at 
least one says not a problem on Solaris.  Any at least some are 
obsolete.  Anyway, 67 are marked as having a patch.  Of those, about 25 
have a github PR, the others should have an uploaded .patch or .diff 
that could be made into a PR.


Slicing another way, 20 and 15 are marked as type 'behavior' and 
'compile error'.  A few errors are possible.  Another 11 have no type 
selected.  So I guess there might be say 30 bug issues that affect 
Solaris and that have a patch to review and improve.


So one or more of your people could select issues of interest and help 
get a patch to where they think is it ready to merge.  Then ask if some 
coredev that can do so will do a final review and possible merge.


In message msg380008 yesterday, Victor listed steps toward a buildbot, 
starting with a census of failing tests to discover the current state of 
CPython 3.10 on Solaris.


--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IMNECV4FMXN274QZHLESIDF5URG7ZVAB/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PyPy performance stats (was Re: Speeding up CPython)

2020-10-26 Thread Terry Reedy

On 10/26/2020 11:42 AM, Matti Picus wrote:


On 10/21/20 2:38 PM, Matti Picus wrote:



[0] https://speed.pypy.org/comparison/


Just as a follow up: the front page of speed.pypy.org now shows the 
latest pypy 3.6 vs cpython 3.6.7.


I just clicked the link and there is 3.7.6, not 3.6.7.  But why not 
current cpython?


Since all executables other than pypy 3.6 are checked, no comparisons 
are shown, and the chart would be impossible.


The default should be just two executables checked.  I am not going to 
try to uncheck 50.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JJ7DOSJBQ25E7C44JFVKYODYCZ7L3A6P/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Speeding up CPython

2020-10-20 Thread Terry Reedy

On 10/20/2020 2:49 PM, Dan Stromberg wrote:

I suspect what it needs most is HPY work, which could benefit a lot of 
Python language implementations in the long term:

https://github.com/hpyproject/hpy

$2e6 spent on HPY could be pretty amazing.


I don't think the two projects are mutually exclusive. I would like to 
see the PSF raise and spend a few million a year on improvements.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/QXYIROIWMSZ4ZHF23C5TCHO2DFLQVZAZ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Performance benchmarks for 3.9

2020-10-14 Thread Terry Reedy

On 10/14/2020 9:16 AM, Pablo Galindo Salgado wrote:


You can check these benchmarks I am talking about by:

* Go here: https://speed.python.org/comparison/
* In the left bar, select "lto-pgo latest in branch '3.9'" and "lto-pgo 
latest in branch '3.8'"


At the moment, there are only results for 'speed-python', none for 
'Broadwell-EP'.  What do those terms mean?


If one leaves all 5 versions checked, they are mis-ordered 3.9, 3.7, 
3.8, 3.6, master.  The correct sequence, 3.6 to master, would be easier 
to read and interpret.  Then pick colors to maximize contrast between 
adjacent bars.



* To better read the plot, I would recommend to select a "Normalization" 
to the 3.8 branch (this is in the top part of the page)


Or either end of whatever sequence one includes.


    and to check the "horizontal" checkbox.


Overall, there have been many substantial improvements since 3.6.

--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DDQC76IF5DF335MTYNCGN7OAFJQPYZUI/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PEP 617 -- New PEG parser for CPython

2020-10-07 Thread Terry Reedy

On 10/6/2020 2:02 PM, Guido van Rossum wrote:
That's appreciated, but I think what's needed more is someone who 
actually wants to undertake this project. It's not just a matter of 
running a small script for hours -- someone will have to come up with a 
way to fuzz that is actually useful for this particular situation 
(inserting random characters in files isn't going to be very effective). 


Changes should be by token or broader grammatical construct.  However, 
the real difficulty in auto testing is the lack of a decision mechanism 
for correct output (code object versus SyntaxError) other than the tests 
being either designed or checked by parser experts.


Classical fuzzing looks for some clearly wrong -- a crash -- rather than 
an answer *or* an Exception.  So yes, random fuzzing that does not pass 
known limits could be done to look for crashes.  But this is different 
from raising SyntaxError to reject wrong programs.


Consider unary prefix operators:

*a is a SyntaxError, because the grammar circumscribes the use of '*' as 
prefix.


-'' is not, which might surprise some, but I presume the error not being 
caught until runtime, as a TypeError, is correct for the grammar as 
written.  Or did I just discover a parser bug?  Or a possible grammar 
improvement?
(In other words, if I, even with my experience, tried grammar/parser 
fuzzing, I might be as much a nuisance as a help.)


It would not necessarily be a regression if the grammar and parser were 
changed so that an obvious error like "- " were to be 
caught as a SyntaxError.




"(One area we have not explored extensively is rejection of all
wrong programs. 


I consider false rejection to be a bigger sin than false acceptance. 
Wrong programs, like "-''", are likely to fail at runtime anyway.  So 
one could test acceptance of randomly generated correct but not 
ridiculously big programs.  But I guess compiling the stdlib and other 
packages already pretty well covered this.



 We have unit tests that check for a certain number
of explicit rejections, but more work could be done, e.g. by using a
fuzzer that inserts random subtle bugs into existing code. We're
open to help in this area.)"
https://www.python.org/dev/peps/pep-0617/#validation



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FNMPQUDPZTX7E4CAGDENNFU6AQJJMW34/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PR checks hang because travis does not report back to github

2020-10-04 Thread Terry Reedy

On 10/4/2020 2:32 PM, Mariatta wrote:
This is a known issue and I have brought it up in GitHub OS Maintainers 
Feedback Group. It happens to other projects as well.


Currently we have branch protection rule where even administrators 
couldnt merge the PR unless all the required checks passed.


Perhaps we can relax the rule to allow administrators to merge the stuck 
PRs. At least temporarily until Travis/GitHub fixes it. Does this sound 
okay?


If we are told how to ping the admins, it would be better than being 
stuck.



On Sun, Oct 4, 2020, 10:44 AM Stefan Behnel > wrote:


Hi devs,

I have a trivial documentation PR

https://github.com/python/cpython/pull/22464

for which travis (unsurprisingly) had a successful run,

https://travis-ci.com/github/python/cpython/builds/187435578


Since the details were still available, I verified 'success'.  This is 
much better that previous stuck situations in which the details line 
disappeared.



but github lists the travis build as "created" instead of "passed".

https://github.com/python/cpython/pull/22464/checks?check_run_id=1188595760

I already tried closing the PR and reopening it, and also triggering the
build again on travis side, but github still fails to pick up the
build status.

I tried creating a new PR, but it seems that github (or travis)
deduplicate
the build requests and still refer to the original build, so that
there is
still no response from travis.

I also cannot find a way to terminate the checks process in github, or
otherwise make it stop waiting for Godot.

Is this a known issue? Is there anything I can do about it?


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IQKOZ62XLWX5Q2YS5B65RFNLQKAKFA4L/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: docs: I'd like new features to references their PEPs

2020-09-14 Thread Terry Reedy

On 9/14/2020 5:25 AM, Cameron Simpson wrote:

On 14Sep2020 01:16, Ned Deily  wrote:

I'll make some PRs. How to submit? Here, or a BPO or something?


My suggestion would be to open one BPO issue for "adding PEP references to 
documentation" and then creating PRs as needed against it.  As you probably know, 
the devguide has the details including for the inline markup role :pep:.

https://devguide.python.org/documenting/#rest-inline-markup


I agree with one master issue.  I would prefer short blurb. "Add 
rationale for __length_hint__ and link to PEP ###.


bpo lists you as triager only, not committer.  If so, I will help review 
and merge if review is requested on PR.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/CXBEWAYSCAZCU7QABRBTKNVPDM3LELUM/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: pty2

2020-09-07 Thread Terry Reedy

On 9/6/2020 10:38 PM, Soumendra Ganguly wrote:

Hello.
I am currently using a tiling window manager ( i3wm ). While
using pty.spawn(), resizing xterm's X window also resizes the
underlying terminal size; as a result, output of commands such as
ls(1) become incorrectly laid out, making them very hard to read.

Further, I have noticed that the pty library has several long-standing
issues (such as bpo-26228).

I have made several improvements to the pty and tty libraries. I
propose that we create a new library "pty2" (
https://github.com/8vasu/pypty2 ) while keeping "pty" for backward
compatibility, because there are several unmerged conflicting PRs
related to these issues.


The problem is that pty has no active maintainer to make decisions. 
Proposing a duplicate module will not solve that.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/37WAPSDMRHHY66IR2POQCCBLEEL6XAWN/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: PR stuck in Travis

2020-08-21 Thread Terry Reedy

On 8/21/2020 2:54 PM, Guido van Rossum wrote:

Does closing and reopening the PR work?



https://github.com/python/cpython/pull/21466


Yes, ready to merge.


--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2EQADIL4XRVTFL2YAUVBG73P5U3MK4GA/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: Procedure for trivial PRs

2020-08-13 Thread Terry Reedy

On 8/13/2020 4:56 PM, Mariatta wrote:

  when landed remove the
"need backport tags" you added...


If done correctly, the "needs backport .." labels got removed 
automatically. We have detailed info here: 
https://devguide.python.org/committing/#backporting-changes-to-an-older-version


The backport bot occasionally fails, sometimes for external reasons.  If 
you hit a problem, you can ask me for help as I have probably hit all 
the failure modes by now.



  a) is it ok to touch 3.9, as it's in rc1?


Yeah bug fixes are accepted to the maintenance branches. I think your PR 
does count as documentation bug fix, so it should be ok to backport to 3.9


Release candidates are done from a separate release branch.  I believe 
that commits now to 3.9 will only appears in 3.9.0rc2 (or 3.9.0) if the 
release manager cherry-picks them into the 3.9.0 release branch. 
Otherwise, 3.9.1.  Doc fixes usually appear online within a day, so that 
pulling them into the release branch should only affects the frozen 
offline copy.



b) shall I create two brand new branches for those PRs, or there's
some way in github/git to "just propose this same change to the other
branches"?

In most cases the bot can handle it simply by adding the "needs backport 
" label. But for more complicated situations, please use the 
cherry_picker.py command line tool.
The documentation on how to do it is here: 
https://github.com/python/cherry-picker#cherry-picking


AFAIK, every PRs is done from its own branch.  Backports are done on 
usually shortlived branches.  Cherry-picker names them 
backport-XXX-3.x, where XXX is the first 7 hex digits of the 
commit and 'x' is the target version.  But that has no significance to 
github.



--
Terry Jan Reedy

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/B52253RAXOI3AV24TI7BPM76WXYBK3NQ/
Code of Conduct: http://python.org/psf/codeofconduct/


[Python-Dev] Re: How about copying the typing module docs from 3.10 to 3.9?

2020-08-11 Thread Terry Reedy

On 8/11/2020 7:59 PM, Luciano Ramalho wrote:


I reorganized the typing module docs, Guido made suggestions, reviewed
and merged it to master.

Right now everything in typing.rst [1] applies to 3.9 as well as 3.10.

[1] https://docs.python.org/3.10/library/typing.html

How about copying the typing.rst file to the 3.9 branch, so more
people would benefit from it sooner?


Doc improvements are routinely backported to versions they apply to, the 
same as with bugfixes.  Doing so is usually decided and done by the 
person merging to master.  The main reason not to do so is when there 
are major merge conflicts.


Doc changes show up online the next day.  I assume that the changes did 
not make the 3.9.0rc1 release ea rlier today.  Whether the release crew 
will update the offline docs included with Windows and Mac installers in 
.0rc2 and .0 is a separate issue from doing a backport.



--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/THV2YUBZHUKRWXIKNMPCKAJTWGVNYUAV/
Code of Conduct: http://python.org/psf/codeofconduct/


  1   2   3   4   5   6   7   8   9   10   >