date:20211230



Raymond Hettinger  added the comment:

I'm thinking of this an a replacement example:

>>> import json
>>> import urllib.request
>>> url = "http://worldtimeapi.org/api/timezone/America/Los_Angeles;
>>> with urllib.request.urlopen(url) as response:
... timeinfo = json.load(response)
...
>>> print(timeinfo['datetime'])
2021-12-30T21:32:08.588278-08:00

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()



Tim Peters  added the comment:

I'm not rejecting anything. Mark wrote the function, and I'll defer to him.

Yes, you added `initial` to `accumulate()`, and I spent hours in all explaining 
to you why it was an utterly natural addition, conspicuous by absence, 
including writing up several real-life use cases that just happened to pop up 
while that discussion dragged on. Thanks! I've used it many more times since it 
was added. Adding it brought accumulate() into line with other languages' 
workalikes. Python was the oddball there.

OTOH, not even gmp's mpz supports rounded isqrt directly, just truncated, and a 
_different_ function that returns not only isqrt(n), but also "the remainder" 
(n - isqrt(n)**2). The latter shows they're not averse to adding cruft that can 
easily be done - which makes it all the more suggestive that they _didn't_ also 
add other rounding modes.

Note: the primary intended "use case" for the version with the remainder 
appears to be a fast way to detect if the argument was a perfect square. As 
mentioned earlier, the lack of a builtin way to do that is the only complaint 
I've seen before about Python's isqrt.

Mark didn't mention his use case for rounded isqrt, I don't have any myself, 
and your only named use case was "building a table of square roots incorporated 
in arbitrary precision functions implemented with scaled integer arithmetic". 
It's very hard to believe that minor speed boosts make any difference at all 
when building static tables, so "it's faster built in" falls flat for me.

To the contrary, adding _any_ optional argument (be it a string or of any other 
type) complicates the calling sequence and requires burning cycles on every 
invocation just to work out which arguments _were_ passed. So, despite that I 
have no expectation of ever asking for a rounded isqrt, adding the possibility 
would cost _me_ cycles on every isqrt() call I already have. None of which are 
in one-shot table-builders, but mostly in loops.

If there's a _compelling_ use case for rounded isqrt, I haven't seen one, but 
I'd be much happier to see it added it as a new function of its own. Like, 
e.g., gmp added mpz_sqrtrem() rather than add an annoying "mode" argument to 
the widely used mpz_sqrt().

I don't really object to string arguments per se, but in the context of what 
one hopes to be blazing fast integer functions, throwing in string comparisons 
just to set a flag seems ludicrously expensive. _PyUnicode_EqualToASCIIId() is 
not my idea of "cheap" ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()



Raymond Hettinger  added the comment:

So, are you going to reject my feature request?  I don't understand why.  Both 
Mark and I have had valid use cases.  The implementation is straight-forward 
and simple.  The workaround is slow and non-obvious.  The default remains the 
same so that usability isn't impaired in any way.  There is basically zero 
downside other that feeling icky about mode flags in general.

With respect to itertools, I did add *initial* to accumulate() solely because 
you wanted it ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Jason R. Coombs  added the comment:


New changeset 35628e4cde71e54afe12aea50c74069afe2c3389 by Jason R. Coombs in 
branch 'main':
bpo-46118: Make sure importlib.resources is included. (GH-30311)
https://github.com/python/cpython/commit/35628e4cde71e54afe12aea50c74069afe2c3389


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37043] Buildbots fail when new files are added



Jason R. Coombs  added the comment:

I encountered this issue again today in issue46118.

I started looking into creating the patchcheck, but I realize it may be a 
little tricky to detect the introduction of a new folder, because `git diff` 
doesn't actually report new folders, and my initial research indicates it's 
[far from straightforward](https://stackoverflow.com/a/36293191/70170).

I thought maybe git diff --dirstat might help, but it doesn't. It reports 100% 
different for a new file in a new folder and same for a new file in an existing 
folder.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Jason R. Coombs  added the comment:

And the underlying cause is reported in issue37043.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Jason R. Coombs  added the comment:

Looks like I encountered the same foot gun when originally introducing the 
importlib.metadata package (issue34632).

--
stage: patch review -> resolved

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Change by Jason R. Coombs :


--
pull_requests: +28525
stage: resolved -> patch review
pull_request: https://github.com/python/cpython/pull/30311

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Jason R. Coombs  added the comment:

Installed buildbots are failing (like this one: 
https://buildbot.python.org/all/#/builders/350/builds/1164). I'm going to see 
if it's a simple fix like updating the makefile or if a rollback is called for.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46109] Separate resources docs from other importlib docs



Jason R. Coombs  added the comment:


New changeset 99945c6b5cf280bd90075cffae942af44941abcc by Jason R. Coombs in 
branch 'main':
bpo-46109: Separate out files relating to importlib.resources (GH-30160)
https://github.com/python/cpython/commit/99945c6b5cf280bd90075cffae942af44941abcc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Change by Jason R. Coombs :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46118] Migrate importlib.resources into a package



Jason R. Coombs  added the comment:


New changeset e712a5b277866a71c195f38c1b5d87d9126dba3e by Jason R. Coombs in 
branch 'main':
bpo-46118: Move importlib.resources to its own package. (#30176)
https://github.com/python/cpython/commit/e712a5b277866a71c195f38c1b5d87d9126dba3e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()



Tim Peters  added the comment:

Pretend this were the itertools module and you were responding to someone 
suggesting a new optional argument to one of its functions. You have tons of 
experience rejecting those, and much the same arguments apply, from "not every 
one-line function needs to be built in" to "in seventeen centuries this is the 
first time someone asked for it" ;-)

Seriously, the only complaint I've seen about isqrt() before is that there 
isn't a version that raises an exception if the argument isn't a perfect 
square. That's so niche I wouldn't support adding that to the core either. 
Sure, sometimes that's what someone may want, and that's fine - they can do it 
on their own.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46178] Remove `.travis.yml`?

2021-12-30 Thread Mariatta



Mariatta  added the comment:


New changeset 2cf7d02b99ce8cebd26d330aa8aac2ee369d4600 by Hugo van Kemenade in 
branch 'main':
bpo-46178: Remove/rename redundant Travis CI code (#30309)
https://github.com/python/cpython/commit/2cf7d02b99ce8cebd26d330aa8aac2ee369d4600


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46210] print deadlocks

2021-12-30 Thread Jason Wilkes



Change by Jason Wilkes :


--
keywords: +patch
pull_requests: +28524
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30310

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()



Raymond Hettinger  added the comment:

> I'd similarly prefer to see recipes in the docs.

Can you elaborate on why you prefer having this in the docs rather than as 
built-in functionality?

For me, the rounded case would be the most common.  I don't think I'm 
better-off writing a wrapper with a semi-opaque trick, having to give it a 
different name and having it run more slowly than built in functionality. Also 
as builtin, I can be assured that it is maintained, documented, and well-tested.

Writing "y = isqrt(x, 'rounded')" is clear, fast, and convenient. I don't see 
the advantage is putting that functionality a little more out of reach.

The string options have been successful elsewhere in Python. For example, 
str.encode() has the error modes: "strict", "ignore", and "replace" which as 
clear, fast, and much better than users having to write their own wrappers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46210] print deadlocks

2021-12-30 Thread Jason Wilkes



New submission from Jason Wilkes :

Hi there. :)

(First time posting here, so apologies in advance if I'm a goof.)

I recently found a deadlock in the stdout and stderr buffer objects.

Following the dev guide, I

(1) wrote up a patch

(2) checked that it fixes the deadlock (on Linux and OS X)

(3) made sure it doesn't break any of the cpython tests (on Linux and OS X)

But I could use some feedback to make sure the patch isn't doing anything dumb, 
and to make sure I'm not missing something obvious.

Here's the situation.

The deadlock only occurs in the sinful state of threading while forking. (I'm 
aware of the reasons why one should never do that.)

But asking newcomers to never mix the two is sort of asking them to know the 
implementation details of every library they use (these abstractions are 
leaky). And I like teaching people about python and concurrency, so it would be 
nice if they didn't have to know all that.

So I've recently developed the ill-advised hobby of trying to run as much code 
as possible in the sinful state of threading while forking, to try to 
understand which bits of the interpreter and standard library are responsible 
for most of the deadlocks, in practice.

All the non-obvious deadlocks I've encountered seemed to be due to one thing, 
namely stdout and stderr's buffer->lock not having an _at_fork_reinit. So if a 
thread is printing during a fork, the child process can never print, else 
deadlock. Standard hazard of threading and forking.

What surprised me was that there didn't seem to be 100s of places in the 
interpreter where threading and forking don't mix. After fixing this one, all 
the different thread + fork code I'd been testing worked.

Which makes me wonder if there's a reason this hasn't been fixed yet.

Hence, maybe the patch is doing something dumb.

Anyways, this is sort of a bug report, and sort of a request for comments on 
how I can make this patch better. Equally cool would be an explanation of why 
re-initializing that lock is a bad idea to begin with.

Thanks for your patience if you read all that.

--
components: IO
files: example.py
messages: 409394
nosy: notarealdeveloper
priority: normal
severity: normal
status: open
title: print deadlocks
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9
Added file: https://bugs.python.org/file50532/example.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Raymond Hettinger  added the comment:

> I'd be happy to change the implementation to use the trailing
> zero counts as suggested.

Thanks.  I think that is a portability win and will made the code a lot easier 
to explain.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22833] The decode_header() function decodes raw part to bytes or str, depending on encoded part

2021-12-30 Thread Daniel Lenski



Daniel Lenski  added the comment:

Due to this bug, any user of this function in Python 3.0+ *already* has to be 
able to handle all of the following outputs in order to use it reliably:

   decode_header(...) -> [(str, None)]
or decode_header(...) -> [(bytes, str)]
or decode_header(...) -> [(bytes, (str|None)), (bytes, (str|None)), ...]

== Fix str/bytes inconsistency ==

We could eliminate the inconsistency, and make the function only ever return 
bytes instead of str, with the following changes to 
https://github.com/python/cpython/blob/3.10/Lib/email/header.py.

```
diff --git a/Lib/email/header.py.orig b/Lib/email/header.py
index 4ab0032..41e91f2 100644
--- a/Lib/email/header.py
+++ b/Lib/email/header.py
@@ -61,7 +61,7 @@ _max_append = email.quoprimime._max_append
 def decode_header(header):
 """Decode a message header value without converting charset.
 
-Returns a list of (string, charset) pairs containing each of the decoded
+Returns a list of (bytes, charset) pairs containing each of the decoded
 parts of the header.  Charset is None for non-encoded parts of the header,
 otherwise a lower-case string containing the name of the character set
 specified in the encoded string.
@@ -78,7 +78,7 @@ def decode_header(header):
 for string, charset in header._chunks]
 # If no encoding, just return the header with no charset.
 if not ecre.search(header):
-return [(header, None)]
+return [header.encode(), None)]
 # First step is to parse all the encoded parts into triplets of the form
 # (encoded_string, encoding, charset).  For unencoded strings, the last
 # two parts will be None.
```

With these changes, decode_header() would return one of the following:

   decode_header(...) -> [(bytes, None)]
or decode_header(...) -> [(bytes, str)]
or decode_header(...) -> [(bytes, (str|None)), (bytes, (str|None)), ...]


== Ensure that charset is always str, never None ==

A couple more small changes:

```
@@ -92,7 +92,7 @@ def decode_header(header):
 unencoded = unencoded.lstrip()
 first = False
 if unencoded:
-words.append((unencoded, None, None))
+words.append((unencoded, None, 'ascii'))
 if parts:
 charset = parts.pop(0).lower()
 encoding = parts.pop(0).lower()
@@ -133,7 +133,8 @@ def decode_header(header):
 # Now convert all words to bytes and collapse consecutive runs of
 # similarly encoded words.
 collapsed = []
-last_word = last_charset = None
+last_word = None
+last_charset = 'ascii'
 for word, charset in decoded_words:
 if isinstance(word, str):
 word = bytes(word, 'raw-unicode-escape')
```

With these changes, decode_header() would return only:

   decode_header(...) -> List[(bytes, str)]

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33937] test_socket: SendmsgSCTPStreamTest.testSendmsgTimeout() failed on Travis CI with: [Errno 12] Cannot allocate memory



Change by Hugo van Kemenade :


--
nosy: +hugovk
nosy_count: 2.0 -> 3.0
pull_requests: +28523
pull_request: https://github.com/python/cpython/pull/30309

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46178] Remove `.travis.yml`?



Change by Hugo van Kemenade :


--
nosy: +hugovk
nosy_count: 2.0 -> 3.0
pull_requests: +28522
pull_request: https://github.com/python/cpython/pull/30309

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22833] The decode_header() function decodes raw part to bytes or str, depending on encoded part

2021-12-30 Thread Daniel Lenski


Daniel Lenski  added the comment:

I recently ran into this bug as well.

For those looking for a reliable workaround, here's an implementation of a 
'decode_header_to_string' function which should Just Work™ in all possible 
cases:

#!/usr/bin/python3
import email.header

# Workaround for https://bugs.python.org/issue22833
def decode_header_to_string(header):
'''Decodes an email message header (possibly RFC2047-encoded)
into a string, while working around 
https://bugs.python.org/issue22833'''

return ''.join(
alleged_string if isinstance(alleged_string, str) else 
alleged_string.decode(
alleged_charset or 'ascii')
for alleged_string, alleged_charset in 
email.header.decode_header(header))


for header in ('=?utf-8?B?ZsOzbw==',
   '=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?=',
   'bar=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?=',
   'plain string',):
print("Header value: %r" % header)
print("email.header.decode_header(...) -> %r" % 
email.header.decode_header(header))
print("decode_header_to_string(...)-> %r" % 
decode_header_to_string(header))
print("---")

Outputs:

Header value: '=?utf-8?B?ZsOzbw=='
email.header.decode_header(...) -> [('=?utf-8?B?ZsOzbw==', None)]
decode_header_to_string(...)-> '=?utf-8?B?ZsOzbw=='
---
Header value: '=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?='
email.header.decode_header(...) -> [(b'hello', 'ascii'), (b'f\xc3\xb3o', 
'utf-8')]
decode_header_to_string(...)-> 'hellofóo'
---
Header value: 'bar=?ascii?Q?hello?==?utf-8?B?ZsOzbw==?='
email.header.decode_header(...) -> [(b'bar', None), (b'hello', 'ascii'), 
(b'f\xc3\xb3o', 'utf-8')]
decode_header_to_string(...)-> 'barhellofóo'
---
Header value: 'plain string'
email.header.decode_header(...) -> [('plain string', None)]
decode_header_to_string(...)-> 'plain string'
---
Header value: 'foo=?blah?Q??='
email.header.decode_header(...) -> [(b'foo', None), (b'', 'blah')]
decode_header_to_string(...)-> 'foo'
---

--
nosy: +dlenski

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46208] os.path.normpath change between 3.11.0a2 and 3.11.0a3+



Hugo van Kemenade  added the comment:

git bisect between v3.11.0a2 and v3.11.0a3 points to the same commit, 
99fcf1505218464c489d419d4500f126b6d6dc28.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46207] Log emit performance degradation in RotatingFileHandlers due to filesystem checks

2021-12-30 Thread Zachary Ware



Change by Zachary Ware :


--
components: +Library (Lib)
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46178] Remove `.travis.yml`?

2021-12-30 Thread Mariatta



Mariatta  added the comment:


New changeset 82c2b54a35f85d87639201d1431ae6584b079e13 by Nikita Sobolev in 
branch 'main':
bpo-46178: remove unusued `.travis.yml` file (#30257)
https://github.com/python/cpython/commit/82c2b54a35f85d87639201d1431ae6584b079e13


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46209] add documentation for decoding newlines in the `io` module

2021-12-30 Thread Guo Ci Teo



Change by Guo Ci Teo :


--
keywords: +patch
pull_requests: +28521
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30308

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46209] add documentation for decoding newlines in the `io` module

2021-12-30 Thread Guo Ci Teo



New submission from Guo Ci Teo :

add documentation for
`io.IncrementalNewlineDecoder`
`io.TextIOWrapper`
`io.StringIO`,
listing various ways to decode newlines

--
messages: 409388
nosy: guoci
priority: normal
severity: normal
status: open
title: add documentation for decoding newlines in the `io` module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46184] Remove `netlify.toml`?

2021-12-30 Thread Mariatta



Mariatta  added the comment:


New changeset fbaf2e604cd354f1ebc6be029480010c6715a8ca by Nikita Sobolev in 
branch 'main':
bpo-46184: remove `netlify.toml` (#30272)
https://github.com/python/cpython/commit/fbaf2e604cd354f1ebc6be029480010c6715a8ca


--
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46197] ensurepip bootstrap breaks out of isolated environment

2021-12-30 Thread kcdodd



Change by kcdodd :


--
keywords: +patch
pull_requests: +28520
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30307

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46208] os.path.normpath change between 3.11.0a2 and 3.11.0a3+

2021-12-30 Thread Christian Heimes



Christian Heimes  added the comment:

The issue could be related to 99fcf1505218464c489d419d4500f126b6d6dc28 / 
bpo-45582.

--
nosy: +christian.heimes, steve.dower

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45569] Drop support for 15-bit PyLong digits?



Mark Dickinson  added the comment:

Terry:

> create a fake test file test/test_xintperf [...]

Sounds reasonable, though I'm not sure I know what exact timings I'd want to 
try. Maybe some of the stock integer-heavy Python benchmarks (pidigits, etc.).

I realised that I have no idea whether any of the buildbots actually use 15-bit 
digits. I've created PR GH-30306 to find out.

--
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45569] Drop support for 15-bit PyLong digits?



Change by Mark Dickinson :


--
keywords: +patch
pull_requests: +28519
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30306

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46208] os.path.normpath change between 3.11.0a2 and 3.11.0a3+



New submission from Hugo van Kemenade :

The behaviour of os.path.normpath appears to have changed between Python 
3.10/Python 3.11.0a2 and 3.11.0a3+.

I don't see anything mentioned in 
https://docs.python.org/3.11/whatsnew/3.11.html

Is this intentional?


Old behaviour:

Python 3.10.1 (v3.10.1:2cd268a3a9, Dec  6 2021, 14:28:59) [Clang 13.0.0 
(clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import normpath
>>> normpath('handbook/../../Tests/image.png')
'../Tests/image.png'
>>>


Python 3.11.0a2 (main, Dec 30 2021, 21:22:15) [Clang 13.0.0 
(clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import normpath
>>> normpath('handbook/../../Tests/image.png')
'../Tests/image.png'
>>>


New behaviour:

Python 3.11.0a3+ (heads/main:af6b406, Dec  9 2021, 15:34:48) [Clang 13.0.0 
(clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import normpath
>>> normpath('handbook/../../Tests/image.png')
'Tests/image.png'
>>>


Python 3.11.0a3+ (heads/main:8d7644f, Dec 30 2021, 21:32:51) [Clang 13.0.0 
(clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os.path import normpath
>>> normpath('handbook/../../Tests/image.png')
'Tests/image.png'
>>>

(Found in https://github.com/sphinx-doc/sphinx/issues/10030.)

--
components: Library (Lib)
messages: 409384
nosy: hugovk
priority: normal
severity: normal
status: open
title: os.path.normpath change between 3.11.0a2 and 3.11.0a3+
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Change by Mark Dickinson :


Added file: https://bugs.python.org/file50531/driver.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Mark Dickinson  added the comment:

Thanks Tim for spotting the stupid mistake. The reworked timings are a bit more 
... plausible.

tl;dr: On my machine, Raymond's suggestion gives a 2.2% speedup in the case 
where POPCNT is not available, and a 0.45% slowdown in the case that it _is_ 
available. Given that, and the fact that a single-instruction population count 
is not as readily available as I thought it was, I'd be happy to change the 
implementation to use the trailing zero counts as suggested.

I'll attach the scripts I used for timing and analysis. There are two of them: 
"timecomb.py" produces a single timing. "driver.py" repeatedly switches 
branches, re-runs make, runs "timecomb.py", then assembles the results.

I ran the driver.py script twice: once with a regular `./configure` step, and 
once with `./configure CFLAGS="-march=haswell"`. Below, "base" refers to the 
code currently in master; "alt" is the branch with Raymond's suggested change 
on it.

Output from the script for the normal ./configure

Mean time for base: 40.130ns
Mean for alt: 39.268ns
Speedup: 2.19%
Ttest_indResult(statistic=7.9929245698581415, pvalue=1.4418376402220854e-14)

Output for CFLAGS="-march=haswell":

Mean time for base: 39.612ns
Mean for alt: 39.791ns
Speedup: -0.45%
Ttest_indResult(statistic=-6.75385578636895, pvalue=5.119724894191512e-11)

--
Added file: https://bugs.python.org/file50530/timecomb.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Serhiy Storchaka  added the comment:

PR 30305 applies Mark's algorithm for larger n (up to 127) depending on k, as 
was suggested by Raymond. Note that it uses different table for limits, which 
maps k to maximal n.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46207] Log emit performance degradation in RotatingFileHandlers due to filesystem checks

2021-12-30 Thread David Fritz



New submission from David Fritz :

I believe this also impacts 3.10 and 3.11 based on the original bpo-45401 which 
led to this change. Prior to commit ac421c348b in the 3.9 branch there were no 
additional os.path checks in the shouldRollover() methods of 
RotatingFileHandler and TimedRotatingFileHandler, which led to maximum 
performance regardless of where log files existed. With the new 
"os.path.exists(self.baseFilename) and not os.path.isfile(self.baseFilename)" 
checks added to these functions, a varying performance degradation is seen if 
the log lives on a filesystem that is not a local disk. It is hard for me to 
strictly call this a "regression", since I understand the reason of the 
original change (only rollover regular files), but I feel this is something 
developers should be aware of, or possibly have a way to disable (without 
making their own subclass) since this cost was not incurred before.

For example, let's say we have user home folders as mounts via S3FS, and each 
user has a log stored in their home folders. Since there are now os.path checks 
on every single emit call, performance of logging is now directly impacted by 
networking latency if using one of these native logging handlers. This came as 
quite a surprise when upgrading to 3.9.8+, when operations that were previously 
taking ~2 seconds were now taking ~2 1/2 minutes. This is reproducible (once 
you have a similar external filesystem mounted) using the following script 
(also attached):


# vi logtest.py
import logging
import logging.handlers

logging.basicConfig(filename="test.log", level=logging.INFO)
logger = logging.getLogger("test_logger")
log_handler = logging.handlers.TimedRotatingFileHandler("test.log", when='D', 
backupCount=2)
logger.addHandler(log_handler)

for i in range(0, 1):
logger.info("iteration: %d", i)


As seen between the commits, cost spikes due to the new calls (double the 
amount of log emit calls):

$ time /build_397/bin/python3 -m cProfile logtest.py |awk '$5>0.00'
...[truncated output to highest percall items]...
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  12/10.0000.0000.5490.549 {built-in method builtins.exec} # 
3.9.7_d57d33c234 (no posix.stat calls)
  12/10.0000.000   70.125   70.125 {built-in method builtins.exec} # 
3.9.7_ac421c348b
 20100   68.9720.003   68.9720.003 {built-in method posix.stat}# 
3.9.7_ac421c348b

This performance degradation is further compounded if for example you have 
logging being performed inside threads, where locks have to be obtained. This 
is actually how I noticed it originally, because I was attempting to use the 
cProfile module to find what was taking so long. I ended up testing several 
python versions to make sure I was not crazy, before identifying the exact 
commit where the change in performance occurred.

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
670.6610.0100.6610.010 {method 'acquire' of '_thread.lock' 
objects} # 3.7.8
671.2370.0181.2370.018 {method 'acquire' of '_thread.lock' 
objects} # 3.9.0
671.2930.0191.2930.019 {method 'acquire' of '_thread.lock' 
objects} # 3.9.4
670.7560.0110.7560.011 {method 'acquire' of '_thread.lock' 
objects} # 3.9.6
671.2180.0181.2180.018 {method 'acquire' of '_thread.lock' 
objects} # 3.9.7
67  152.4112.275  152.4112.275 {method 'acquire' of '_thread.lock' 
objects} # 3.9.8
67  152.3822.274  152.3822.274 {method 'acquire' of '_thread.lock' 
objects} # 3.9.9

Ultimately I do not have a recommendation for a native fix that keeps both the 
improvement from the commit, and the performance from before the commit. 
Perhaps caching the results of the checks until there is a rollover? Only 
perform the check on initial handler creation? Otherwise it appears the only 
solution is to subclass and override the shouldRollover() to not include the 
os.path checks and match <=3.9.7 behavior. This is the solution I am 
implementing in the codebase/environment where this was discovered while this 
report is discussed further. This feels less than ideal, but if that is the 
consensus then hopefully this report can help the next person to come across 
it. Thank you for your time and all the hard work put into Python!

--
files: logtest.py
messages: 409381
nosy: dfritz
priority: normal
severity: normal
status: open
title: Log emit performance degradation in RotatingFileHandlers due to 
filesystem checks
type: performance
versions: Python 3.10, Python 3.11, Python 3.9
Added file: https://bugs.python.org/file50529/logtest.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe:

[issue37295] Possible optimizations for math.comb()



Change by Serhiy Storchaka :


--
pull_requests: +28518
pull_request: https://github.com/python/cpython/pull/30305

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46206] Crash when editing emoji containing strings

2021-12-30 Thread M Z



M Z <10mauryc...@gmail.com> added the comment:

FYI: My platform is arch linux on amd64.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46206] Crash when editing emoji containing strings

2021-12-30 Thread M Z


New submission from M Z <10mauryc...@gmail.com>:

Reproduction steps:
0. start rpel. command: ``python``
1. enter into  '___'
2. use arrow keys and backspace to delete an underscore
3. press enter.
4. observe segfault

- insertion of chars before emoji can also cause crash

back trace (gdb):

```
(gdb) bt
#0  0x77495d22 in raise () from /lib/libc.so.6
#1  0x7747f862 in abort () from /lib/libc.so.6
#2  0x7747f747 in __assert_fail_base.cold () from /lib/libc.so.6
#3  0x7748e616 in __assert_fail () from /lib/libc.so.6
#4  0x777e7b6e in get_error_line (p=p@entry=0x76e53b80, 
lineno=lineno@entry=0) at Parser/pegen.c:438
#5  0x77b3660d in _PyPegen_raise_error_known_location 
(p=p@entry=0x76e53b80, errtype=errtype@entry=0x77e7ac00 
<_PyExc_SyntaxError>, lineno=0, col_offset=0, end_lineno=0, end_col_offset=-1, 
errmsg=0x77c84bcd "(%s) %U",
va=0x7fffdcb0) at Parser/pegen.c:491
#6  0x77b36d33 in _PyPegen_raise_error (p=p@entry=0x76e53b80, 
errtype=0x77e7ac00 <_PyExc_SyntaxError>, errmsg=errmsg@entry=0x77c84bcd 
"(%s) %U") at Parser/pegen.c:422
#7  0x77b37213 in raise_decode_error (p=p@entry=0x76e53b80) at 
Parser/pegen.c:271
#8  0x77bbed14 in initialize_token (token_type=60, end=0x0, 
start=, token=0x76ffb330, p=0x76e53b80) at 
Parser/pegen.c:712
#9  _PyPegen_fill_token (p=p@entry=0x76e53b80) at Parser/pegen.c:785
#10 0x77c5eca1 in statement_newline_rule (p=0x76e53b80) at 
Parser/parser.c:1521
#11 interactive_rule (p=0x76e53b80) at Parser/parser.c:994
#12 _PyPegen_parse (p=p@entry=0x76e53b80) at Parser/parser.c:33180
#13 0x77bd5784 in _PyPegen_run_parser (p=p@entry=0x76e53b80) at 
Parser/pegen.c:1343
#14 0x77bd8e72 in _PyPegen_run_parser_from_file_pointer 
(fp=fp@entry=0x77619800 <_IO_2_1_stdin_>, start_rule=start_rule@entry=256, 
filename_ob=filename_ob@entry=0x77155490, enc=enc@entry=0x7714bdb0 
"utf-8",
ps1=ps1@entry=0x77155c40 ">>> ", ps2=ps2@entry=0x76ffafa0 "... ", 
flags=0x7fffe0f8, errcode=0x7fffdfe4, arena=0x76feb820) at 
Parser/pegen.c:1440
#15 0x77bd903e in _PyParser_ASTFromFile (fp=fp@entry=0x77619800 
<_IO_2_1_stdin_>, filename_ob=filename_ob@entry=0x77155490, 
enc=enc@entry=0x7714bdb0 "utf-8", mode=mode@entry=256, 
ps1=ps1@entry=0x77155c40 ">>> ",
ps2=ps2@entry=0x76ffafa0 "... ", flags=0x7fffe0f8, 
errcode=0x7fffdfe4, arena=0x76feb820) at Parser/peg_api.c:26
#16 0x77bd9205 in PyRun_InteractiveOneObjectEx 
(fp=fp@entry=0x77619800 <_IO_2_1_stdin_>, 
filename=filename@entry=0x77155490, flags=flags@entry=0x7fffe0f8) at 
Python/pythonrun.c:257
#17 0x77bda02d in _PyRun_InteractiveLoopObject (fp=0x77619800 
<_IO_2_1_stdin_>, filename=0x77155490, flags=0x7fffe0f8) at 
Python/pythonrun.c:148
#18 0x77bdbbc7 in _PyRun_AnyFileObject (fp=0x77619800 
<_IO_2_1_stdin_>, filename=0x77155490, closeit=0, flags=0x7fffe0f8) at 
Python/pythonrun.c:84
#19 0x77bdbdf3 in PyRun_AnyFileExFlags (fp=0x77619800 
<_IO_2_1_stdin_>, filename=, closeit=0, flags=0x7fffe0f8) at 
Python/pythonrun.c:116
#20 0x77bddf60 in pymain_run_stdin (config=0xf910) at 
Modules/main.c:502
#21 pymain_run_python (exitcode=0x7fffe0f0) at Modules/main.c:590
#22 Py_RunMain () at Modules/main.c:666
#23 0x77bde683 in pymain_main (args=args@entry=0x7fffe250) at 
Modules/main.c:696
#24 0x77bde7f9 in Py_BytesMain (argc=argc@entry=1, 
argv=argv@entry=0x7fffe3a8) at Modules/main.c:720
#25 0x60af in main (argc=1, argv=0x7fffe3a8) at 
./Programs/python.c:15
(gdb)
```

--
components: Unicode
messages: 409379
nosy: 10maurycy10, ezio.melotti, vstinner
priority: normal
severity: normal
status: open
title: Crash when editing emoji containing strings
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46195] Annotated and Optional get_type_hints buggy interaction

2021-12-30 Thread Nikita Sobolev



Change by Nikita Sobolev :


--
keywords: +patch
pull_requests: +28517
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30304

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46070] _PyImport_FixupExtensionObject() regression causing a crash in subintepreters

2021-12-30 Thread Manuel Reimer



Change by Manuel Reimer :


--
nosy: +M-Reimer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46202] remove opcode POP_EXCEPT_AND_RERAISE

2021-12-30 Thread Brandt Bucher



Change by Brandt Bucher :


--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()



Tim Peters  added the comment:

[Mark]
> The idea is no more than: "compute an extra bit, then use
> that extra bit to determine which way to round".

Thanks! Despite that this is what the derivation I sketched boiled down to, I 
was still missing how general the underlying principle was. Duh! Indeed, 
nothing special about sqrt here. So I'm delightfully surprised again :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Tim Peters  added the comment:

> Aargh! That is of course what I meant, but not in fact
> what I timed. :-(

!!! Even more baffling then. Seems like the code posted got out of 
math_comb_impl() early here:

if (overflow || ki > ni) {
result = PyLong_FromLong(0);
goto done;
}

67 out of every 68 times comb() was called, before any actual ;-) computation 
was even tried. Yet one way was significantly faster than the other overall, 
despite that they were so rarely executed at all?

Something ... seems off here ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Mark Dickinson  added the comment:

> I'm assuming you meant to write comb(67, k) instead

Aargh! That is of course what I meant, but not in fact what I timed. :-(

I'll redo the timings. Please disregard the previous message.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46195] Annotated and Optional get_type_hints buggy interaction

2021-12-30 Thread Guido van Rossum



Guido van Rossum  added the comment:

Yes, we changed PEP 484 in https://github.com/python/peps/pull/689.

So get_type_hints() should follow suit.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46205] Race condition in runtest_mp leads to hangs (never exits)

2021-12-30 Thread Sam Gross



Change by Sam Gross :


--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46205] Race condition in runtest_mp leads to hangs (never exits)

2021-12-30 Thread Sam Gross



Change by Sam Gross :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46205] Race condition in runtest_mp leads to hangs (never exits)

2021-12-30 Thread Sam Gross



New submission from Sam Gross :

The runtest_mp.py has a race condition between checking for worker.is_alive() 
and processing the queue that can lead to indefinite hangs.

The hang happens when the all the results from the self.output queue are 
processed but at least one of the workers hasn't finished exiting.

https://github.com/python/cpython/blob/8d7644fa64213207b8dc6f555cb8a02bfabeced2/Lib/test/libregrtest/runtest_mp.py#L394-L418

The main thread tries to get a result from the output queue, but the queue is 
empty and remains empty. Although the queue.get() operation eventually times 
out (after 30 seconds), the main thread does not re-check if all the workers 
have exited (!), but instead retries the queue.get() in the "while True" loop.

https://github.com/python/cpython/blob/8d7644fa64213207b8dc6f555cb8a02bfabeced2/Lib/test/libregrtest/runtest_mp.py#L415-L418

To reproduce, apply the below patch which introduces a small delay to more 
reliably trigger the hang.

curl 
"https://gist.githubusercontent.com/colesbury/fe3769f43dfb724c86ecbb182b1f6749/raw/e29a4eaeebb8d5252cdd66f3f8a70f7bc5fa14e7/runtest_mp.diff;
 | patch -p1
./python -m test test_regrtest -m test_module_from_test_autotest -v

--
messages: 409374
nosy: colesbury
priority: normal
severity: normal
status: open
title: Race condition in runtest_mp leads to hangs (never exits)
versions: Python 3.10, Python 3.11, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Tim Peters  added the comment:

[Mark]
> I ran some timings for comb(k, 67) on my macOS / Intel MacBook Pro,
> using timeit to time calls to a function that looked like this:
>
> def f(comb):
> for k in range(68):
> for _ in range(256):
> comb(k, 67)
> comb(k, 67)
>... # 64 repetitions of comb(k, 67) in all

I'm assuming you meant to write comb(67, k) instead, since the comb(k, 67) 
given is 0 at all tested k values except for k=67, and almost never executes 
any of the code in question.

It's surprising to me that even the long-winded popcount code was faster! The 
other way needs to read up 3 1-byte values from a trailing zero table, but the 
long-winded popcount emulation needs to read up 4 4-byte mask constants (or are 
they embedded in the instruction stream?), in addition to doing many more 
bit-fiddling operations (4 shifts, 4 "&" masks, 3 add/subtract, and a multiply 
- compared to just 2 add/subtract).

So if the results are right, Intel timings make no sense to me at all ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45033] Calls to PyErr_PrintEx in destructors cause calling async functions to incorrectly return None

2021-12-30 Thread Zac Bentley



Zac Bentley  added the comment:

For context (copied from the linked GitHub issue), this has affected at least 
two OSS projects:

https://github.com/apache/pulsar/issues/11823
https://github.com/tbodt/v8py/issues/45

The proposed alternative ("PyErr_Fetch, then PyErr_Display, then 
PyErr_Restore") does not work well: if code runs in a boost-python destructor 
which runs during async return time, how would it tell the difference between a 
"real" exception raised from within whatever function is currently returning 
and the "fake" StopIteration exception that is in the error global while an 
async function returns? Aka, this code would not have any way to function 
properly:

async def func():
thing = MyNativeCodeObject()
raise ValueError()

The destructor for MyNativeCodeObject would have to perform a complex check, 
and PyErr_Restore/do nothing IFF the following are all true:
- Is there an exception currently being handled/in the global?
- Is that exception a StopIteration?
- Is the interpreter currently in the process of returning from an async 
function (I'm not sure how to check this)?


The fact that non-StopIteration exceptions are handled correctly (e.g. the 
ValueError in the above example makes it out of the async call during "await 
func()") means, I think, that there's already some secondary exception storage 
mechanism that's used to stash and restore user exceptions while the async 
machinery raises StopIteration to return from an async function. Could that 
same mechanism be used to "hide" the StopIteration from user code that runs 
during async-return time?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46196] documentation for cmd library should include columnize() function

2021-12-30 Thread Nikita Sobolev



Change by Nikita Sobolev :


--
keywords: +patch
nosy: +sobolevn
nosy_count: 2.0 -> 3.0
pull_requests: +28516
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30303

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45033] Calls to PyErr_PrintEx in destructors cause calling async functions to incorrectly return None

2021-12-30 Thread Zac Bentley



Zac Bentley  added the comment:

Irit, the documentation is clear.

However, the problem is that the presence of StopIteration in the exception 
global during async return time causes very unexpected behavior.

If, during async return time (in the linked issue, I do this in a boost-python 
object destructor), I write code like this:

try {
boost::python::call(somefunction);
} catch (boost::python::error_already_set e) {
PyErr_Print();
}

then two things go wrong:
1. The "catch" block runs even if "somefunction" did not raise, because 
StopIteration is already in the exception handler.
2. When PyErr_Print clears StopIteration, the currently-returning async 
function returns None, regardless of what it was supposed to return. I consider 
this to be interpreter-induced data corruption.

I'm not sure what happens if user code raises an additional exception.

Do you think this is a Python bug, or a Boost bug? I'm not familiar enough with 
interpreter internals to know whether or not there are other ways besides 
boost-python destructors to get code to run "during async return". If there 
are, I would definitely consider this a Python bug: PyErr_Print should not be 
able to corrupt interpreter/return state.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46204] Graphlib documentation (general cleanup)

2021-12-30 Thread David Mc Dougall



Change by David Mc Dougall :


--
keywords: +patch
pull_requests: +28515
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30269

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46204] Graphlib documentation (general cleanup)

2021-12-30 Thread David Mc Dougall



New submission from David Mc Dougall :

The graphlib documentation has some grammar & phrasing issues.

--
components: Library (Lib)
messages: 409370
nosy: dam1784
priority: normal
severity: normal
status: open
title: Graphlib documentation (general cleanup)
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46014] functools.singledispatch does not support Union types

2021-12-30 Thread Alex Waygood



Alex Waygood  added the comment:

```
>>> from typing import Optional, get_origin
>>> get_origin(Optional[int])
typing.Union
```

^Because of that, it will work with typing.Optional as well as typing.Union and 
types.UnionType, yes.

I am planning on submitting a docs PR at some point in the next few days 
(probably linked to a new BPO issue).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46071] Graphlib documentation (edge direction)

2021-12-30 Thread David Mc Dougall



Change by David Mc Dougall :


--
title: Graphlib documentation -> Graphlib documentation (edge direction)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35166] BUILD_MAP_UNPACK doesn't function as expected for dict subclasses



Irit Katriel  added the comment:

BUILD_MAP_UNPACK and BUILD_MAP_UNPACK_WITH_CALL have been removed.

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46192] Optimize builtin functions min() and max()

2021-12-30 Thread Erlend E. Aasland



Erlend E. Aasland  added the comment:

Repeating my comment on GH-30286: If we are touching `min()` and `max()`, it 
would make sense, IMO, to port them to Argument Clinic. FTR, Argument Clinic 
got `*args` support in GH-18609 / bpo-20291.

@colorfulappl made a "competing" branch using AC on his local fork[^1]. 
However, that branch contained a bug with the key function; I made an amended 
version[^2] for benchmarking. Here's some micro-benchmarks from optimised 
builds on macOS 12.1 w/Clang 13:

```
+-+-+---+--+
| Benchmark   | main| GH-30286  
| GH-30286-ac  |
+=+=+===+==+
| max(a, b)   | 193 ns  | 74.1 ns: 
2.60x faster | 179 ns: 1.08x faster |
+-+-+---+--+
| max(a, b, c, d, e)  | 273 ns  | 126 ns: 
2.17x faster  | 260 ns: 1.05x faster |
+-+-+---+--+
| max([a, b]) | 267 ns  | 185 ns: 
1.44x faster  | 239 ns: 1.12x faster |
+-+-+---+--+
| max([a, b, c, d, e])| 345 ns  | 259 ns: 
1.33x faster  | 312 ns: 1.10x faster |
+-+-+---+--+
| max((a,), (b,), key=lambda x: x[0]) | 707 ns  | 444 ns: 
1.59x faster  | 513 ns: 1.38x faster |
+-+-+---+--+
| max((a,), (b,), (c,), (d,), (e,), key=lambda x: x[0])   | 1.12 us | 831 ns: 
1.35x faster  | 930 ns: 1.20x faster |
+-+-+---+--+
| max([(a,), (b,)], key=lambda x: x[0])   | 786 ns  | 561 ns: 
1.40x faster  | 570 ns: 1.38x faster |
+-+-+---+--+
| max([(a,), (b,), (c,), (d,), (e,)], key=lambda x: x[0]) | 1.19 us | 981 ns: 
1.22x faster  | 981 ns: 1.22x faster |
+-+-+---+--+
| max([], default=-1) | 484 ns  | 177 ns: 
2.73x faster  | 188 ns: 2.57x faster |
+-+-+---+--+
| Geometric mean  | (ref)   | 1.68x 
faster  | 1.29x faster |
+-+-+---+--+
```



[^1]: 
https://github.com/colorfulappl/cpython/commit/29b9559de31ae19e8d127d7a3063494b2d9791b0
[^2]: https://github.com/erlend-aasland/cpython/tree/gh-30286-ac

--
nosy: +erlendaasland
Added file: https://bugs.python.org/file50528/bench-minmax.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46014] functools.singledispatch does not support Union types

2021-12-30 Thread wrobell



wrobell  added the comment:

Will it support Optional as well?

According to PEP-604, these two shall be equivalent (using Python 3.10 below)?
```
type(int | None)
types.UnionType

type(tp.Optional[int])
typing._UnionGenericAlias
```

Also, IMHO, the documentation of singledispatch should be improved. It needs to 
state that only Python types are supported with few exceptions (union type 
now), but other annotation types are not supported, i.e. generics like 
`list[int]`.

--
nosy: +wrobell

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45569] Drop support for 15-bit PyLong digits?



Mark Dickinson  added the comment:

I posted a request for information on usage of 15-bit digits to python-dev: 
https://mail.python.org/archives/list/python-...@python.org/thread/ZICIMX5VFCX4IOFH5NUPVHCUJCQ4Q7QM/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1598083] Top-level exception handler writes to stdout unsafely



Irit Katriel  added the comment:

I think this can be closed as out of date because PyFile_WriteString/Object no 
longer use puts and they do check for errors. The error checking of 
PyFile_WriteString/Object in the traceback display code was the subject of 
Issue45635.

--
nosy: +iritkatriel
resolution:  -> out of date
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45033] Calls to PyErr_PrintEx in destructors cause calling async functions to incorrectly return None



Irit Katriel  added the comment:

Zac, the documentation for PyErr_Print [1] states that it clears the error 
indicator. This is akin to handling the exception. Was this your intention?

If you want to print the exception without clearing it, I think you would 
probably need to call PyErr_Fetch, then PyErr_Display, then PyErr_Restore.

[1] https://docs.python.org/3/c-api/exceptions.html#c.PyErr_PrintEx

--
nosy: +iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46202] remove opcode POP_EXCEPT_AND_RERAISE



Change by Irit Katriel :


--
keywords: +patch
pull_requests: +28514
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30302

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()


Mark Dickinson  added the comment:

> I'd be happy to see recipes added to the docs for rounded and ceiling flavors 
> of isqrt, but am dubious about the value of building them in.

I'd similarly prefer to see recipes in the docs. We already have such a recipe 
for ceil(√n).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46187] Optionally support rounding for math.isqrt()


Mark Dickinson  added the comment:

> did you invent this?

The idea is no more than: "compute an extra bit, then use that extra bit to 
determine which way to round". More generally, for any real number x, the 
nearest integer to x (rounding ties towards +infinity) is `⌊(⌊2x⌋ + 1) / 2⌋`. 
Now put x = √n, then ⌊2x⌋ is ⌊√(4n)⌋, and the rest follows.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37295] Possible optimizations for math.comb()



Mark Dickinson  added the comment:

> So which of xor-popcount and add-up-up-trailing-zero-counts is faster may 
> well depend on platform.

I ran some timings for comb(k, 67) on my macOS / Intel MacBook Pro, using 
timeit to time calls to a function that looked like this:

def f(comb):
for k in range(68):
for _ in range(256):
comb(k, 67)
comb(k, 67)
... # 64 repetitions of comb(k, 67) in all

Based on 200 timings of this script with each of the popcount approach and the 
uint8_t-table-of-trailing-zero-counts approach (interleaved), the popcount 
approach won, but just barely, at around 1.3% faster. The result was 
statistically significant (SciPy gave me a result of 
Ttest_indResult(statistic=19.929941828072433, pvalue=8.570975609117687e-62)).

Interestingly, the default build on macOS/Intel is _not_ using the dedicated 
POPCNT instruction that arrived with the Nehalem architecture, presumably 
because it wants to produce builds that will still be useable on pre-Nehalem 
machines. It uses Clang's __builtin_popcount, but that gets translated to the 
same SIMD-within-a-register approach that we have already in pycore_bitutils.h.

If I recompile with -msse4.2, then the POPCNT instruction *is* used, and I get 
an even more marginal improvement: a 1.7% speedup over the lookup-table-based 
version.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46203] Add timeout for Windows build steps



Change by Mark Dickinson :


--
keywords: +patch
pull_requests: +28513
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30301

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46203] Add timeout for Windows build steps



New submission from Mark Dickinson :

Recently there was an upstream issue with GitHub Actions that caused the 
Windows build steps in build.yml to hang. No output for the step was displayed 
in the build logs until the entire job was eventually cancelled, after the 
default step timeout of 6 hours.

I don't know how to fix the "no output" problem, but we can mitigate the 6 hour 
wait by adding a timeout for the build step. Some external discussion suggested 
that a conservative timeout of 30 minutes would be appropriate; looking at 
recent PRs, the build usually completes in around 5 minutes.

The (soon-to-be-)attached PR adds that timeout.

Here's the log from one of the failed jobs: 
https://github.com/python/cpython/runs/4641823914?check_suite_focus=true  (note 
that this link will probably eventually become invalid).

Here's the relevant GitHub incident: 
https://www.githubstatus.com/incidents/gh0vvxtlj5v7

--
messages: 409359
nosy: christian.heimes, mark.dickinson, steve.dower
priority: normal
severity: normal
status: open
title: Add timeout for Windows build steps
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34498] Python 3.7+ break on singledispatch_function.register(pseudo_type), which Python 3.6 accepted

2021-12-30 Thread wrobell



wrobell  added the comment:

To add bit more context, there is a discussion about Python types and
annotation types, where suggestion of closing this bug has been raised

https://github.com/python/mypy/issues/9773

The discussion is about the need of distinction between Python types and
annotation types, and explicit terminology in the documentation.

Closing this bug gives clearer indication to tools like Mypy, how to deal
with annotation types used with singledispatch.

IMHO, it is small step towards more explict Python type system, whatever
its limitations.

--
nosy: +wrobell

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46195] Annotated and Optional get_type_hints buggy interaction

2021-12-30 Thread Nikita Sobolev

Nikita Sobolev added the comment:

As Guido said, the root cause of this problem is because `None` default
automatically adds `Optional` to the resulting type.

Source:
https://github.com/python/cpython/blob/8d7644fa64213207b8dc6f555cb8a02bfabeced2/Lib/typing.py#L1854-L1856

So, what happens there:
- correct `value` is passed to `_eval_type`, correct result
`typing.Annotated[typing.Optional[str], 'd']` is returned at this point
- then `if name in defaults and defaults[name] is None:` adds extra `Optional`
annotation on top of `Annotated`

> in the past a default of None automatically caused an Optional
to be added, but we changed our minds

Guido, are you talking about https://github.com/python/typing/issues/275 ?

Now all type-checkers (AFAIK) support something similar to
`--no-implicit-optional` mode.

Having this in mind, I see different solutions to the current problem:
1. Remove `Optional` inference with `None` default. This is going to be a
some-what breaking change. The only positive side of this is that we can really
simplify our code (mainly because the other solution is to complicate our code
even more).
2. Or we can change this place to explicitly check for `Annotated` type and its
internal type. This should be the easiest to write and backport. But, it still
has some complexity to it. I think that this is a better solution: we don't
break existing behavior, change is local and pretty trivial.

Also caused by this:
- https://bugs.python.org/issue42921
- https://bugs.python.org/issue42288

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46202] remove opcode POP_EXCEPT_AND_RERAISE



New submission from Irit Katriel :

Following issue45711 this opcode can be replaced by 

ADDOP(c, ROT_THREE);
ADDOP(c, ROT_THREE);
ADDOP(c, POP_EXCEPT);
ADDOP_I(c, RERAISE, 1);

--
assignee: iritkatriel
components: Interpreter Core
messages: 409356
nosy: iritkatriel
priority: normal
severity: normal
status: open
title: remove opcode POP_EXCEPT_AND_RERAISE
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46199] Calculation influenced by print

2021-12-30 Thread Steven D'Aprano



Steven D'Aprano  added the comment:

Please try to provide a minimal reproducible example:

https://stackoverflow.com/help/minimal-reproducible-example

http://sscce.org/

At the very least, you need to provide three pieces of information:

1. What you tried.
2. What you expected to happen (and why, if it isn't obvious).
3. What actually happened instead.


Ideally you should be able to get your example down to using one or two values, 
not three lists with 30 values each (90 values).

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46201] PEP 495 misnames PyDateTime_DATE_GET_FOLD

2021-12-30 Thread Carl Drougge



New submission from Carl Drougge :

PEP 495 names one of the accessor macros PyDateTime_GET_FOLD but the code names 
it PyDateTime_DATE_GET_FOLD.

The FOLD macros are also missing from 
https://docs.python.org/3/c-api/datetime.html (and versions).

--
assignee: docs@python
components: Documentation
messages: 409354
nosy: docs@python, drougge
priority: normal
severity: normal
status: open
title: PEP 495 misnames PyDateTime_DATE_GET_FOLD
type: behavior
versions: Python 3.10, Python 3.11, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46200] Discourage logging f-strings due to security considerations

2021-12-30 Thread Eric V. Smith



Change by Eric V. Smith :


--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46189] Text containing "wide" character not correctly refreshed



Change by Serhiy Storchaka :


--
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34498] Python 3.7+ break on singledispatch_function.register(pseudo_type), which Python 3.6 accepted