date:20220216



Guido van Rossum  added the comment:

> > I note that there is no documented way to retrieve the cancel message

> Does retrieving it from the CancelledError that is bubbling up suffice? Or do 
> you need to be able to obtain it from the future object?

I'm not sure yet (if anything I'd need it for a task, not a future). But it's 
also not documented that it gets passed to the exception (at least not in the 
Task docs -- I didn't check the Future docs).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

2022-02-16 Thread Chris Jerdonek



Chris Jerdonek  added the comment:

> I note that there is no documented way to retrieve the cancel message

Does retrieving it from the CancelledError that is bubbling up suffice? Or do 
you need to be able to obtain it from the future object?

--
nosy: +chris.jerdonek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46745] Typo in new PositionsIterator


Jelle Zijlstra  added the comment:


New changeset 482a4b6c3f8ef60056e85647a1d84af2e6dfd3ef by Robert-André Mauchin 
in branch 'main':
bpo-46745: Fix typo in PositionsIterator (#31322)
https://github.com/python/cpython/commit/482a4b6c3f8ef60056e85647a1d84af2e6dfd3ef


--
nosy: +Jelle Zijlstra

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46763] os.path.samefile incorrect results for shadow copies

2022-02-16 Thread Eryk Sun



Eryk Sun  added the comment:

Sample implementation:

import os
import msvcrt
import win32file 

def samefile(f1, f2):
"""Test whether two paths refer to the same file or directory."""
s1 = os.stat(f1)
s2 = os.stat(f2)
return _common_same_file(f1, f2, s1, s2)


def sameopenfile(fd1, fd2):
"""Test whether two file descriptors refer to the same file."""
s1 = os.fstat(fd1)
s2 = os.fstat(fd2)
return _common_same_file(fd1, fd2, s1, s2)


def _common_same_file(f1, f2, s1, s2):
if s1.st_ino != s2.st_ino or s1.st_dev != s2.st_dev:
return False

# (st_dev, st_ino) may be insufficient on its own. Use the final
# NT path of each file to refine the comparison.
p = _get_final_nt_paths([f1, f2])

# The stat result is unreliable if the volume serial number (st_dev)
# or file ID (st_ino) is 0.
if 0 in (s1.st_dev, s1.st_ino):
if None in p:
return False
return p[0] == p[1]

# A volume shadow copy has the same volume serial number as the
# base volume. In this case, the device names have to be compared.
d = _get_device_names(p)
if any('volumeshadowcopy' in n for n in d if n):
return d[0] == d[1]

return True


def _get_final_nt_paths(files):
result = []
nt_normal = 0x2 # VOLUME_NAME_NT | FILE_NAME_NORMALIZED
nt_opened = 0xA # VOLUME_NAME_NT | FILE_NAME_OPENED
for f in files:
p = None
if f is not None:
try:
p = _getfinalpathname(f, nt_normal)
except OSError:
try:
p = _getfinalpathname(f, nt_opened)
except OSError:
pass
result.append(p)
return result


def _get_device_names(paths):
# Look for "\Device\{device name}[\]".
result = []
for p in paths:
d = None
if p is not None:
q = p.split('\\', 3)
if len(q) > 2 and q[1].lower() == 'device' and q[2]:
d = q[2].lower()
result.append(d)
return result


def _getfinalpathname(p, flags=0):
try:
if isinstance(p, int):
h = msvcrt.get_osfhandle(p)
else:
h = win32file.CreateFile(p, 0, 0, None, win32file.OPEN_EXISTING,
win32file.FILE_FLAG_BACKUP_SEMANTICS, None)
return win32file.GetFinalPathNameByHandle(h, flags)
except win32file.error as e:
strerror = e.strerror.rstrip('\r\n .')
raise OSError(0, strerror, p, e.winerror) from None

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

2022-02-16 Thread Christoph Reiter



Change by Christoph Reiter :


--
nosy:  -lazka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Dong-hee Na  added the comment:


New changeset 8cb5f707a841832aeac4b01c8b6a0e72dced52ae by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from array module (GH-31376)
https://github.com/python/cpython/commit/8cb5f707a841832aeac4b01c8b6a0e72dced52ae


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46333] ForwardRef.eq does not respect module parameter



Change by Jelle Zijlstra :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46775] [Windows] OSError should unconditionally call winerror_to_errno

2022-02-16 Thread Eryk Sun



New submission from Eryk Sun :

bpo-37705 overlooked fixing the OSError constructor. oserror_parse_args() in 
Objects/exceptions.c should unconditionally call winerror_to_errno(), which is 
defined in PC/errmap.h. 

winerror_to_errno() maps the Winsock range 1-11999 directly, except for the 
6 errors in this range that are based on C errno values: WSAEINTR, WSAEBADF, 
WSAEACCES, WSAEFAULT, WSAEINVAL, and WSAEMFILE. Otherwise, Windows error codes 
that aren't mapped explicitly get mapped by default to EINVAL.

--
components: Interpreter Core, Windows
keywords: easy (C)
messages: 413383
nosy: eryksun, paul.moore, steve.dower, tim.golden, zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: [Windows] OSError should unconditionally call winerror_to_errno
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46333] ForwardRef.eq does not respect module parameter



miss-islington  added the comment:


New changeset 29ae7d3996e073ec3f15c4e7486b17c1d0e3c8f5 by Miss Islington (bot) 
in branch '3.9':
bpo-46333: Honor `module` parameter in ForwardRef (GH-30536)
https://github.com/python/cpython/commit/29ae7d3996e073ec3f15c4e7486b17c1d0e3c8f5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46333] ForwardRef.eq does not respect module parameter



miss-islington  added the comment:


New changeset a17d59a6df146077b5420b36c35a1b82ab3f071a by Miss Islington (bot) 
in branch '3.10':
[3.10] bpo-46333: Honor `module` parameter in ForwardRef (GH-30536) (GH-31379)
https://github.com/python/cpython/commit/a17d59a6df146077b5420b36c35a1b82ab3f071a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46333] ForwardRef.eq does not respect module parameter



Change by miss-islington :


--
pull_requests: +29530
pull_request: https://github.com/python/cpython/pull/31380

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46333] ForwardRef.eq does not respect module parameter



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 8.0 -> 9.0
pull_requests: +29529
pull_request: https://github.com/python/cpython/pull/31379

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46333] ForwardRef.eq does not respect module parameter



Jelle Zijlstra  added the comment:


New changeset 6e7b813195f9bd6a2a15c1f00ef2c0180f6c751a by aha79 in branch 
'main':
bpo-46333: Honor `module` parameter in ForwardRef (GH-30536)
https://github.com/python/cpython/commit/6e7b813195f9bd6a2a15c1f00ef2c0180f6c751a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46066] Deprecate keyword args syntax for TypedDict definition



Change by Jelle Zijlstra :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46066] Deprecate keyword args syntax for TypedDict definition



Jelle Zijlstra  added the comment:


New changeset de6043e596492201cc1a1eb28038970bb69f3107 by 97littleleaf11 in 
branch 'main':
bpo-46066: Deprecate kwargs syntax for TypedDict definitions (GH-31126)
https://github.com/python/cpython/commit/de6043e596492201cc1a1eb28038970bb69f3107


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29992] Expose parse_string in JSONDecoder

2022-02-16 Thread Inada Naoki



Inada Naoki  added the comment:

> Generally speaking, parsing some things as decimal or datetime are schema 
> dependent.

Totally agree with this.

> In order to provide maximal flexibility it would be much nicer to have a 
> streaming interface available (like SAX for XML parsing), but that is not 
> what this is.

I think it is too difficult and complicated.
I think post-processing approach (e.g. dataclass_json, pydantic) is enough.

--
nosy: +methane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

> (I note that there is no documented way to retrieve the cancel message; 
> you're supposed to access the protected `_cancel_message` attribute, 
> apparently. Looks like we forgot something there.)

Oh, it's passed to the CancelledError() constructor. But that's not documented 
either (I had to find the original issue that introduced this to figure it out 
-- bpo-31033).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

FWIW it looks like this part of taskgroups is vulnerable to a similar race:

https://github.com/python/cpython/blob/6f1efd19a70839d480e4b1fcd9fecd3a8725824b/Lib/asyncio/taskgroups.py#L212-L232

Deleting the two lines I mentioned won't fix it here; a hack using the cancel 
message might be more appropriate. (I note that there is no documented way to 
retrieve the cancel message; you're supposed to access the protected 
`_cancel_message` attribute, apparently. Looks like we forgot something there.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46774] Importlib.metadata.version picks first distribution not latest

2022-02-16 Thread Kevin Kirsche



Change by Kevin Kirsche :


--
nosy: +kkirsche

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46774] Importlib.metadata.version picks first distribution not latest

2022-02-16 Thread Kevin Kirsche



New submission from Kevin Kirsche :

When using importlib.metadata.version with tools such as poetry which may 
install the current package one or more times, importlib.metadata.version is 
not deterministic in returning the latest version of the package, instead 
returning the first one located.

As it's unclear if this behavior is desired by importlib, I'm creating this 
issue to determine if this is intentional behavior or a bug.

I have opened the following poetry issue:
* https://github.com/python-poetry/poetry/issues/5204

I have also created the following reproduction repository for the installation 
issue:
https://github.com/kkirsche/poetry-remove-untracked

When the after is modified to return the version, it returns the first one 
found (e.g. if you go 3.0.0 -> 3.0.1 -> 3.0.2, each would be installed and the 
library would return 3.0.0 to the caller)

Thank you for your time and consideration. I apologize if this is not something 
that requires action by the Python team.

I'd be open to trying to submit a PR, but want to verify whether this is 
intentional or not.

--
components: Library (Lib)
messages: 413375
nosy: kkirsche2
priority: normal
severity: normal
status: open
title: Importlib.metadata.version picks first distribution not latest
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.

2022-02-16 Thread Jason Wilkes



Change by Jason Wilkes :


--
nosy: +notarealdeveloper
nosy_count: 7.0 -> 8.0
pull_requests: +29528
pull_request: https://github.com/python/cpython/pull/30310

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Dong-hee Na  added the comment:

> I think it's fine to use the one issue.

Great! and thank you for all your efforts :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46766] Add a class for file operations so a syntax such as open("file.img", File.Write | File.Binary | File.Disk) is possible.

2022-02-16 Thread Isaac Johnson



Isaac Johnson  added the comment:

Well that is how it works with open. It is implemented in the io module and  
added to builtins.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

Hm, I see. So the problem is that in the interval between move_on's calls to 
t.cancel() and t.uncancel(), if the web server calls t.cancel() that will just 
return False. So the web server would have to implement some other mechanism 
for cancelling operations.

That's indeed unfortunate. Maybe we  should just roll back that aspect of the 
TaskGroup PR -- in particular, remove these two lines:

if self._cancel_requested:
return False

from Task.cancel(). These lines don't matter for TaskGroup (it works without 
them), and they weren't there before yesterday, so the fallout would be very 
localized.

@asvetlov What do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Tin Tvrtković  added the comment:

@Guido Imagine something_else() takes a long time and a lot of server resources 
(like a heavy query). If the web server disconnected a tad later and avoided 
the race condition, the task would have gotten cancelled very soon after the 
start of something_else() and stopped running it. But since the race did 
happen, the query avoids cancellation (potentially ~forever).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

@Tin The sad path is just a race, right? If the web server had disconnected 
just a tad later, __aexit__() would already have returned and await 
something_else() would already be running. So you can't make any promises if 
you write your code that way anyway.

@Alex For "happy eyeballs" you could also raise an exception or cancel the 
parent task once you've saved the winning result somewhere. Maybe you could 
show example code written using different paradigms so we can compare.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Alex Grönholm  added the comment:

I just tried to write a snippet to demonstrate the issue in TaskGroup, but that 
doesn't seem possible since TaskGroup never swallows a CancelledError. It 
always raises/propagates _some_ exception out of __aexit__() unless of course 
all the child tasks run to completion successfully.

I was also surprised to notice that TaskGroup doesn't have a .cancel() method, 
so in cases where one would launch multiple tasks and cancel the rest when one 
succeeds, one would have to store all the child tasks separately and then 
iterate over them and cancel one by one. The Happy Eyeballs algorithm is one 
such use case (also used in AnyIO this way).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46755] QueueHandler logs stack_info twice

2022-02-16 Thread Ned Deily



Change by Ned Deily :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

Tin Tvrtković added the comment:

Ok, first the happy path. We have a task running this:

async def a():
with move_on_after(5):
await a_long_op()
await something_else()

`move_on_after` has scheduled a callback to run that calls
`parent_task.cancel(msg=1)` 5 seconds after it was executed.

So now 5 seconds pass, the callback cancels the task, and `move_on_after`
catches the CancelledError, sees the msg == 1, and swallows it.
`something_else()` now runs. All good.

Sad path. Same scenario, except the event loop is kinda busy since we're
running in production. Turns out this task was spawned by a web server, and
there's a 5 second timeout (or the client disconnected, or something else). So
now we have 2 callbacks that want to cancel this task: the one from
`move_on_after` and the one from the web server.

The one from the web server is more important, since it's a higher level
cancellation. But the callback from `move_on_after` runs first, and marks the
task for cancellation, and sets the message to 1. Then, before the task gets to
run, the webserver also cancels the task. But that does nothing:
https://github.com/python/cpython/blob/6f1efd19a70839d480e4b1fcd9fecd3a8725824b/Lib/asyncio/tasks.py#L206.

So now the task actually gets to run, `move_on_after` swallows the
CancelledError, and something_else() gets to run. But ideally, it shouldn't.

___
Python tracker

___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue43275] [venv] Add a `.gitignore` file with content `*` in the root folder generated by venv module

2022-02-16 Thread Brett Cannon



Change by Brett Cannon :


--
superseder:  -> [venv] Adding a .gitignore file to virtual environments

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

I hesitate to add yet another stack at this fundamental level.

The Task.cancel() method returns a bool which indicates whether any state 
changed.

When multiple cancellations happen concurrently, all but the first will return 
False, and anything that would like to cancel a task but finds that t.cancel() 
returns False can know that the task was either already cancelled or has 
already terminated. (To tell the difference, check t.done() first.)

What would be the use case of wanting to cancel multiple times and having each 
cancellation be delivered separately?

I know of one use case, where a task somehow decides to catch and *ignore* 
CancelledError (this should not be done lightly but it is supported -- like 
shielding in Trio). An impatient user or task manager might want to cancel such 
a thread a second time. This is what .uncancel() is for -- the thread must call 
.uncancel() to signal that it has truly ignored the cancellation (as opposed to 
being busy with cleanup that it deems uncancellable).

But in this case the second cancellation (if it is to have any effect) should 
be delivered *after* .uncancel() is called.

Your proposal of a cancel context or stack seems to be suggesting that there's 
a use case for mutliple *concurrent* cancellations. But I find it difficult to 
imagine such a use case, so I need your help.

Even if we ignore the stack idea, could you provide some code showing how the 
cancel context would be used? I just learn better from code examples.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Tin Tvrtković  added the comment:

On the topic of TaskGroups needing to know whether to swallow a child 
CancelledError or no: just use the same nonce/cancellation context trick?

I remember asking Nathaniel about why in Trio nurseries and cancel scopes were 
linked (nurseries contain a cancel scope there), whereas in Quattro they are 
completely separate, and not really understanding the answer. I think I'm 
getting it now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Tin Tvrtković  added the comment:

The use of the cancel message is a hack, yeah. But it's what I had to work 
with. We could introduce a new, proper cancellation context to Tasks instead, 
which could be attached to the CancelError when it's raised.

On the topic of multiple cancellations being applied to a task before it gets 
to run: could we just treat the cancellation context as a stack, and when the 
task gets to run, we throw them all in, one by one? Other options would involve 
somehow figuring out what the highest priority cancellation context is, or 
combining all the cancellation contexts into the CancelError somehow.

On the topic of a task getting to run at least once before being cancelled: 
sure, I guess. I've personally never needed this but I can see how it'd be 
useful. Either have a flag on the Task instance or build that logic into the 
cancellation context handling?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46750] some code paths in ssl and _socket still import idna unconditionally

2022-02-16 Thread Shivaram Lingamneni



Shivaram Lingamneni  added the comment:

I wanted to check in about the status of this patch. Here's the case for the 
patch, as I understand it:

1. It is not a novel optimization, it just consistently applies design 
decisions that were made previously (RFE #1472176 and bpo-22127).
2. The performance impact of the initial import of encodings.idna and its 
transitive dependencies is in fact macroscopic relative to the baseline costs 
of the interpreter: 5 milliseconds to import the modules and 500 KB in 
increased RSS, relative to baselines of approximately 50 milliseconds to set up 
and tear down an interpreter and 10 MB in RSS.

Here are the relevant benchmarks, first for time:


```python
import time
start = time.time()
'a'.encode('idna')
print(time.time() - start)
```

and for memory:

```python
import os
def rss():
os.system('grep VmRSS /proc/' + str(os.getpid()) + '/status')
rss()
'a'.encode('idna')
rss()
```

Are there potential changes to this patch that would mitigate your concerns?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46756] Incorrect authorization check in urllib.request

2022-02-16 Thread Martin Panter



Martin Panter  added the comment:

Maybe the same as Issue 42766?

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46769] Improve documentation for `typing.TypeVar`



Guido van Rossum  added the comment:

Maybe historically constrained type vars predated bound type vars. This could 
have influenced the way PEP 484 was written and then the docs were derived from 
the PEP.

But yes, this should be fixed in as many places as possible (including the mypy 
docs, probably).

I would even support a minor update to PEP 484 to help people find the right 
way (the PEP is still widely cited). It would have to be just be a textual 
change, not a spec change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

Oh, wait. The new Task.cancelling() API helps tell the difference between the 
*parent* task being cancelled "from outside" vs. the task group itself 
cancelling the parent task so as to break out of an await like the following:

async with TaskGroup() as g:
g.create_task(...)
await 

when the await is cancelled, __aexit__() is called with a CancelledError, and 
we need to tell whether it was cancelled from the outside or by the completion 
callback on one of the tasks managed by the task group.

The EdgeDB TaskGroup monkey-patched the parent task's cancel() method, and the 
new asyncio.TaskGroup instead checks parent.cancelled().

However, AFAICT when *any* of the task managed by the TaskGroup exits with 
CancelledError, this is *ignored* (in both the EdgeDB version and in 
asyncio.TaskGroup). The assumption here seems to be that the only reason a 
managed task raises CancelledError is because it was cancelled by the TaskGroup.

A fix for that would be to separately keep track (maybe in a separate weak 
dict, or some other data structure -- maybe a flag on the task itself?) of 
which tasks are successfully cancelled by the TaskGroup. We can then treat a 
CancelledError bubbling out of a managed task that we *didn't* cancel as any 
other exception, causing it to abort the task group (i.e., cancel all other 
tasks).

Is that what you are looking for? (But I think this could be solved even in 3.9 
without resorting to cancel messages.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

2022-02-16 Thread Irit Katriel



Change by Irit Katriel :


--
nosy: +ajoino

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Alex Grönholm  added the comment:

Thanks, I will take a look at .uncancel() and .cancelling(). I saw that work 
happening in my email feed but couldn't figure out right away how it helped, 
but I will definitely look into the new TaskGroup code to see how it's used 
there and will get back to you after that.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

OK.

1. Please have a look at how .cancelling()/.uncancel() works (there's an 
example of it in TaskGroup) to solve that particular problem without using the 
cancel message.

2. Suppose you could make (backwards compatible) changes to asyncio. What would 
you do? 3.11 feature freeze (aka beta 1) is still a few months away (late May) 
so now's the time to get your wishes in.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Alex Grönholm  added the comment:

I'm not trying to argue that asyncio should be changed to have level 
cancellation or even cancel scopes as built-in (at this point), but expanding 
the low level API to make implementing these features possible in third party 
libraries without the awkward hacks we have now.

As for async-timeout, it suffers from the same problem as AnyIO and Quattro: 
that cancellations of the entire task can be inadvertently swallowed by the 
async context manager in edge cases. I hadn't even thought of the possibility 
of this happening until one of AnyIO's users reported just such a problem: 
https://github.com/agronholm/anyio/issues/374

I just couldn't think of any way to correctly support such things without at 
least _some_ changes to the task cancellation behavior, and allowing .cancel() 
to update the cancel message seemed like the least invasive option. I'm all 
ears if someone has a better solution.

--
nosy:  -ajoino

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

2022-02-16 Thread Jacob Nilsson



Change by Jacob Nilsson :


--
nosy: +ajoino

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46766] Add a class for file operations so a syntax such as open("file.img", File.Write | File.Binary | File.Disk) is possible.

2022-02-16 Thread Steven D'Aprano



Steven D'Aprano  added the comment:

True, but you did say it would be with the io module in your original 
suggestion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29418] inspect.isroutine does not return True for some bound builtin methods

2022-02-16 Thread Hakan Çelik


Change by Hakan Çelik :


--
pull_requests: +29527
pull_request: https://github.com/python/cpython/pull/31377

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes

2022-02-16 Thread Joshua Bronson



Change by Joshua Bronson :


--
nosy: +jab

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

Alex, the goal here is not to replicate every Trio feature or behavior. For 
example I don't think that asyncio is likely to get level cancellation in 3.11, 
but we can certainly see if we can do something about some of the edge cases 
you mention, like the case of a task that is cancelled before it has started 
running, where you say that the task should be allowed to run until its first 
await.

It would be nice to have a native asyncio example that demonstrates this, so we 
have a concrete goal.

I am thinking it is something like this:

async def send_from_open_file(f, s):
data = f.read()
f.close()
await s.send(data)

async def send_filename(filename, s):
f = open(filename)
t = asyncio.create_task(send_from_open_file(f, s))
t.cancel()
await asyncio.sleep(1)

This is an interesting edge case and I can see why you'd rather see this run 
until the `await s.send(data)` line. The question is, can we do that without 
breaking other promises implicit or explicit? (Just because the docs don't 
mention some behavior that doesn't mean we can change it. We have to consider 
what happens to actual real world code.)

I don't even know if this would be easy to change if we decided it was a good 
change. Thoughts? (Possibly this discussion belongs in a new issue, since it's 
not directly related to adding cancel scopes.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Eric Snow  added the comment:

(from https://github.com/python/cpython/pull/31376#issuecomment-1041836106)

[corona10]
> Should we create the separate bpo issue if module changes are too noisy?

I think it's fine to use the one issue.  There are only 26 modules with 
`NEEDS_PY_IDENTIFIER` and none have much risk with the change.  So it's 
unlikely that we'll get any discussion about any specific module.  If we do, we 
can easily create an issue then for the module in question.  If one of the 
modules had many uses of `_Py_IDENTIFIER()` then it might make sense to split 
it out, but IIRC none of them have very many.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Change by Guido van Rossum :


--
nosy: +tinchester

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Alex Grönholm  added the comment:

A brief explanation of cancel scopes for the uninitiated: A cancel scope can 
enclose just a part of a coroutine function, or an entire group of tasks. They 
can be nested within each other (by using them as context managers), and marked 
as shielded from cancellation, which means cancellation won't be propagated 
(i.e. raised in the coroutine function) from a cancelled outer scope until 
either the inner scope's shielding is disabled or the inner scope is exited or 
cancelled directly.

The fundamental problem in implementing these on top of asyncio is that native 
task cancellation can throw a wrench in these gears. Since a cancel scope works 
by catching a cancellation error and then (potentially) allowing the coroutine 
to proceed, it would have to know, when catching a cancellation error, if the 
cancellation error was targeted at a cancel scope or the task itself. A 
workaround for this, made possible in Python 3.9, is to (ab)use cancellation 
messages to include the ID of the target cancel scope. This only solves half of 
the problem, however. If the task is already pending a cancellation targeted at 
a cancel scope, the task itself cannot be cancelled anymore since calling 
cancel() again on the task is a no-op. This would be solved by updating the 
cancel message on the second call. The docs don't say anything about the 
behavior on the second call, so it's not strictly speaking a change in 
documented behavior.

Then, on the subject of level cancellation: level cancellation builds upon 
cancel scopes and changes cancellation behavior so that whenever a task yields 
while a cancelled cancel scope is in effect, it gets hit with a CancelledError 
every time, as opposed to just once in asyncio's "edge" style cancellation. 
Another very important difference is that with level cancellation, even a task 
that starts within a cancelled scope gets to run up until the first yield 
point. This gives it an opportunity to clean up any resources it was given 
ownership of (a connected socket in a socket server is a common, practical 
example of this).

This is what the asyncio documentation states about Task.cancel():

"This arranges for a CancelledError exception to be thrown into the wrapped 
coroutine on the next cycle of the event loop.

The coroutine then has a chance to clean up or even deny the request by 
suppressing the exception with a try … … except CancelledError … finally block. 
Therefore, unlike Future.cancel(), Task.cancel() does not guarantee that the 
Task will be cancelled, although suppressing cancellation completely is not 
common and is actively discouraged."

This is, however, only true for a task that has started running. A Task that 
gets cancelled before even entering the coroutine is silently dropped.

As asyncio does not allow for custom task instances without overriding the 
entire task factory, it leaves libraries like AnyIO some less desirable options 
for implementing level cancellation:

1. Implementing a parallel task system using low level synchronous callbacks 
(con: such tasks won't show up in asyncio.all_tasks() or work with third party 
debugging tools)
2. Adding callbacks to continuously cancel tasks that yield inside a cancelled 
scope (con: ugly; potentially extra overhead?)
3. Adding a wrapper for the task that acts as a "controller" (con: adds an 
extra visible stack frame, messes with the default task name)

Having low level machinery for injecting a custom Task instance to the event 
loop would probably solve this problem.

--
nosy: +alex.gronholm -tinchester

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29418] inspect.isroutine does not return True for some bound builtin methods

2022-02-16 Thread Batuhan Taskaya



Change by Batuhan Taskaya :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics



Guido van Rossum  added the comment:

I have a PR up to typeshed to add the new Task methods and a new stub file 
taskgroups.pyi: https://github.com/python/typeshed/pull/7240

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46772] Statically Initialize PyArg_Parser in clinic.py



Change by Eric Snow :


--
dependencies: +Add a Private API for Looking Up Global Objects, Statically 
allocate and initialize the empty tuple.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46773] Add a Private API for Looking Up Global Objects



New submission from Eric Snow :

We need this to statically initialize PyArg_Parser.kwtuple.  (See bpo-46772.)

For now this will be a "private" API (leading underscore).  Ultimately, we'll 
want a Public API, so we can eventually stop exposing *any* objects as symbols 
in the C-API.  However, that will need a PEP.

--
assignee: eric.snow
components: Interpreter Core
messages: 413352
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Add a Private API for Looking Up Global Objects
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46772] Statically Initialize PyArg_Parser in clinic.py



New submission from Eric Snow :

The code generated by clinic.py is already partially statically initialized.  
Currently we init the other fields in Python/getargs.c:parser_init(), which 
runs the first time we try to use each parser.  AFAICS, that remaining init 
that could be done statically using the data we have available in clinic.py 
during code generation.

My primary interest is in static init of PyArg_Parser.kwtuple, which is a tuple 
containing only strings.

--
assignee: eric.snow
components: Interpreter Core
messages: 413351
nosy: eric.snow
priority: normal
severity: normal
stage: needs patch
status: open
title: Statically Initialize PyArg_Parser in clinic.py
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Guido van Rossum  added the comment:

Sure, we should create the best possible solution.

We have no CI in the stdlib that checks type annotations, so those should 
probably be moved to a stub file in typeshed. (Ditto for asyncio taskgroups.py.)

Using the new .cancelling()/.uncancel() API added to Task you might be able to 
avoid hacks using the cancel msg (check how it's used in the new TaskGroup).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Tin Tvrtković  added the comment:

Oh, and Trio's `current_effective_deadline` is also in.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics

2022-02-16 Thread Yury Selivanov



Yury Selivanov  added the comment:

> There's one thing that gather() does that TaskGroup doesn't: it gives us the 
> return values from the tasks.

That's easy to do with task groups too:

  async with TaskGroup() as g:
  r1 = g.create_task(coro1())
  r2 = g.create_task(coro2())

  print(r1.result())

  # or
  print(await r2)  # I *think* this should work

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes


Tin Tvrtković  added the comment:

I've essentially forked `async-timeout` (a very good library) into Quattro 
cancel scopes: 
https://github.com/Tinche/quattro/blob/main/src/quattro/cancelscope.py.

The differences are:
* the API follows Trio, so instead of `timeout` you'd use `fail_after` or 
`move_on_after`
* instead of `async with timeout`, you use a normal context manager `with 
fail_after`. The Trio folks think this is important (less suspension points, 
less race conditions) and I agree
* it's somewhat composable (as much as possible under asyncio), each scope 
knows if the CancelError is meant for it or should be propagated further. This 
is implemented by using the CancelError message to carry a nonce. This only 
works on 3.9+, but here that's not a problem
* small deadline adjustment differences, I use a setter on the deadline instead 
of `update` and `shift`
* it's fully type annotated, but so is Andrew's

Let me know if this sounds interesting.

--
nosy: +tinchester

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics



Change by Guido van Rossum :


--
nosy: +dhalbert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



Change by Guido van Rossum :


--
keywords: +patch
pull_requests: +29526
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/31270

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics



Guido van Rossum  added the comment:

I've created a separate issue for cancel scopes: bpo-46771.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46771] Add some form of cancel scopes



New submission from Guido van Rossum :

Now that TaskGroup is merged (see bpo-46752) we might consider adding some form 
of cancel scopes (another Trio idea).

There's a sensible implementation we could use as a starting point in 
@asvetlov's async-timeout package (https://github.com/aio-libs/async-timeout).

--
components: asyncio
messages: 413345
nosy: asvetlov, gvanrossum, iritkatriel, njs, yselivanov
priority: normal
severity: normal
stage: needs patch
status: open
title: Add some form of cancel scopes
type: enhancement
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics



Guido van Rossum  added the comment:

@dhalbert, it's probably better to file a new issue if you want changes to 
gather(). Although I suppose that if we want to deemphasize it, we shouldn't be 
adding new features to it. My own new feature idea would be to have it wait for 
all tasks and then if there are any exceptions, raise an ExceptionGroup. That 
(like any new gather() behaviors) would require a new keyword-only flag to 
gather(). If we're going to deemphasize it I might not bother though.

There's one thing that gather() does that TaskGroup doesn't: it gives us the 
return values from the tasks. The question is whether that's useful. If it is 
maybe we should *not* deepmhasize gather() quite as much and then adding new 
features would be okay.

--
nosy:  -dhalbert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46760] test_dis should test the dis module, not everything else



Change by Mark Shannon :


--
nosy: +brandtbucher, iritkatriel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46770] ConfigParser(dict_type=) not behaving as expected

2022-02-16 Thread Mark Lonnemann



New submission from Mark Lonnemann :

ConfigParser() is not using a custom dictionary class correctly, according to 
my understanding.  I have duplicate options in a config file that I want to 
rename uniquely.  The following code does not work.
x = 0
class MultiDict(dict):
def __setitem__(self, key, value):
if key == 'textsize':
global x
key += str(x)
x += 1
dict.__setitem__(self, key, value)

...

config1 = cp.ConfigParser(dict_type=MultiDict)
config1.read('ini_file.ini')

"textsize" is the option named twice in my config file.  When I run the code, I 
get a DuplicateOptionError for "textsize".  No one seems to know how to solve 
this, so it could be a bug.  If it's sloppy coding, I apoligize.

--
components: Extension Modules
messages: 413343
nosy: malonn
priority: normal
severity: normal
status: open
title: ConfigParser(dict_type=) not behaving as expected
type: behavior
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46769] Improve documentation for `typing.TypeVar`

2022-02-16 Thread Alex Waygood



New submission from Alex Waygood :

There are three variants of `TypeVar`s:

(1) TypeVars that are neither constrained nor bound: `T = TypeVar("T")`
(2) TypeVars that are bound: `U = TypeVar("U", bound=str)`
(3) TypeVars that are constrained: `V = TypeVar("V", str, bytes)`

The third variant is important for annotating certain functions, such as those 
in the `re` module. However, it has a number of issues (see 
https://github.com/python/typing/discussions/1080 for further discussion):

(1) It has somewhat surprising semantics in many situations.
(2) It is difficult for type checkers to deal with, leading to a number of bugs 
in mypy, for example.
(3) Many users (especially people relatively inexperienced with Python typing) 
reach for constrained TypeVars in situations where using bound TypeVars or the 
@overload decorator would be more appropriate.

Both PEP 484 and the documentation for the typing module, however:

(1) Give examples for variants (1) and (3), but not for variant (2), which is 
treated as something of an afterthought.
(2) Do not mention that TypeVars can be bound to a union of types, which is an 
important point: `T = TypeVar("T", str, bytes)` has different semantics to `T = 
TypeVar("T", bound=str|bytes)`, and often the latter is more appropriate.

--
assignee: docs@python
components: Documentation
messages: 413342
nosy: AlexWaygood, Jelle Zijlstra, docs@python, gvanrossum, kj, sobolevn
priority: normal
severity: normal
stage: needs patch
status: open
title: Improve documentation for `typing.TypeVar`
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46072] Unify handling of stats in the CPython VM

2022-02-16 Thread Brandt Bucher



Brandt Bucher  added the comment:


New changeset 580cd9ab2992b7df6f4815020b5841e14a5a6977 by Brandt Bucher in 
branch 'main':
bpo-46072: Add detailed failure stats for BINARY_OP (GH-31289)
https://github.com/python/cpython/commit/580cd9ab2992b7df6f4815020b5841e14a5a6977


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46702] Specialize UNPACK_SEQUENCE

2022-02-16 Thread Brandt Bucher



Brandt Bucher  added the comment:


New changeset a9da085015db8cbb81f660158864ac94fe6c67a2 by Brandt Bucher in 
branch 'main':
bpo-46702: Specialize UNPACK_SEQUENCE (GH-31240)
https://github.com/python/cpython/commit/a9da085015db8cbb81f660158864ac94fe6c67a2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46768] CVE-201-4160 Vulnerability Is Found in Lib/site-packages/cryptography/hazmat/bindings/_openssl.pyd for Cryptography Version 3.3.2

2022-02-16 Thread Christian Heimes



New submission from Christian Heimes :

The file is from 3rd party package PyCA cryptography and not maintained by us. 
3.3.2 is an old version of cryptography and no longer supported. I recommend 
that you update to cryptography 36.0.1

--
nosy: +christian.heimes
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Change by Dong-hee Na :


--
pull_requests: +29525
pull_request: https://github.com/python/cpython/pull/31376

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29992] Expose parse_string in JSONDecoder

2022-02-16 Thread Tobias Oberstein



Tobias Oberstein  added the comment:

> It's unlikely that you would want to parse every string that looks enough 
> like a decimal as a decimal, or that you would want to pay the cost of 
> checking every string in the whole document to see if it's a decimal.

fwiw, yes, that's what I do, and yes, it needs to check every string

https://github.com/crossbario/autobahn-python/blob/bc98e4ea5a2a81e41209ea22d9acc53258fb96be/autobahn/wamp/serializer.py#L410

> Returning a decimal as a string is becoming quite common in REST APIs to 
> ensure there is no floating point errors.

exactly. it is simply required if money values are involved.

since JSON doesn't have a native Decimal, strings need to be used (the only 
scalar type in JSON that allows one to encode the needed arbitrary precision 
decimals)

CBOR has tagged decimal fraction encoding, as described in RFC7049 section 
2.4.3.

fwiw, we've added roundtrip and crosstrip testing between CBOR <=> JSON in our 
hacked Python JSON, and it works

https://github.com/crossbario/autobahn-python/blob/bc98e4ea5a2a81e41209ea22d9acc53258fb96be/autobahn/wamp/test/test_wamp_serializer.py#L235

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Dong-hee Na  added the comment:


New changeset e8a19b092fc0551581e10d6f310dd5feabac7609 by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from mmap module (GH-31375)
https://github.com/python/cpython/commit/e8a19b092fc0551581e10d6f310dd5feabac7609


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Change by Dong-hee Na :


--
pull_requests: +29524
pull_request: https://github.com/python/cpython/pull/31375

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46766] Add a class for file operations so a syntax such as open("file.img", File.Write | File.Binary | File.Disk) is possible.

2022-02-16 Thread Isaac Johnson



Isaac Johnson  added the comment:

Well it wouldn't need to be imported. I was working on including it inside 
builtins like open(). It wouldn't be very convenient if it needed to be 
imported.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46752] Introduce task groups to asyncio and change task cancellation semantics

2022-02-16 Thread Dan Halbert



Dan Halbert  added the comment:

For your TODO list (not sure how else to communicate this):

I agree with the de-emphasis of gather(). I think adding another version of 
gather() that cancels all the remaining tasks if one fails would also be good, 
unless you think it is completely redundant due to TaskGroups. This idea was 
originally mentioned in https://bugs.python.org/issue31452 as a bug, and 
determined to be "works as designed". So now making an all-cancel() version of 
gather() is an idiom that people keep recoding, e.g. 
https://stackoverflow.com/questions/59073556/how-to-cancel-all-remaining-tasks-in-gather-if-one-fails.

--
nosy: +dhalbert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Dong-hee Na  added the comment:


New changeset b2077117d125925210148294eefee28797b7ff4c by Erlend Egeberg 
Aasland in branch 'main':
bpo-46541: Replace _Py_IDENTIFIER with _Py_ID in sqlite3 (GH-31351)
https://github.com/python/cpython/commit/b2077117d125925210148294eefee28797b7ff4c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Dong-hee Na  added the comment:


New changeset d64f3caebe8f8e31ecd193e0bd25105400153ece by Dong-hee Na in branch 
'main':
bpo-46541: Remove usage of _Py_IDENTIFIER from csv module (GH-31372)
https://github.com/python/cpython/commit/d64f3caebe8f8e31ecd193e0bd25105400153ece


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46767] [Doc] sqlite3 Cursor.execute() return value is unspecified

2022-02-16 Thread Erlend E. Aasland


Erlend E. Aasland  added the comment:

Quoting the docs:

> This is a nonstandard shortcut that creates a cursor object by calling the
> cursor() method, calls the cursor’s execute() method with the parameters
> given, and returns the cursor.

There is already a backlink (or back reference) to the Connection.cursor() 
method, which again has a backlink to the Cursor class. What would you suggest 
to make things clearer?

How about the following rewording:

"[...] and returns the newly created cursor object".

(IMO this is too verbose; I'm fine with the current docs.)

--
nosy: +erlendaasland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46329] Split up the CALL_NO_KW and CALL_KW instructions.



Change by Mark Shannon :


--
pull_requests: +29523
pull_request: https://github.com/python/cpython/pull/31373

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45069] python 3.9.2 contains libcrypto-1_1.dll and libssl-1_1.dll associates CVE-2021-23840\CVE-2021-3450\CVE-2021-3711\CVE-2021-3712\CVE-2021-23841\CVE-2021-3449 of openssl-1.1.1i

2022-02-16 Thread zjmxq



Change by zjmxq :


--
components:  -Library (Lib)
title: CVE-201-4160 Vulnerability Is Found in 
Lib/site-packages/cryptography/hazmat/bindings/_openssl.pyd for Cryptography 
Version 3.3.2 -> python 3.9.2 contains libcrypto-1_1.dll and libssl-1_1.dll 
associates 
CVE-2021-23840\CVE-2021-3450\CVE-2021-3711\CVE-2021-3712\CVE-2021-23841\CVE-2021-3449
 of openssl-1.1.1i

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46768] CVE-201-4160 Vulnerability Is Found in Lib/site-packages/cryptography/hazmat/bindings/_openssl.pyd for Cryptography Version 3.3.2

2022-02-16 Thread zjmxq



Change by zjmxq :


--
components: Library (Lib)
nosy: zjmxq
priority: normal
severity: normal
status: open
title: CVE-201-4160 Vulnerability Is Found in 
Lib/site-packages/cryptography/hazmat/bindings/_openssl.pyd for Cryptography 
Version 3.3.2
type: security
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45069] CVE-201-4160 Vulnerability Is Found in Lib/site-packages/cryptography/hazmat/bindings/_openssl.pyd for Cryptography Version 3.3.2

2022-02-16 Thread zjmxq



Change by zjmxq :


--
components: +Library (Lib)
title: python 3.9.2 contains libcrypto-1_1.dll and libssl-1_1.dll associates 
CVE-2021-23840\CVE-2021-3450\CVE-2021-3711\CVE-2021-3712\CVE-2021-23841\CVE-2021-3449
 of openssl-1.1.1i -> CVE-201-4160 Vulnerability Is Found in 
Lib/site-packages/cryptography/hazmat/bindings/_openssl.pyd for Cryptography 
Version 3.3.2
type:  -> security

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46763] os.path.samefile incorrect results for shadow copies

2022-02-16 Thread Eryk Sun



Eryk Sun  added the comment:

Python uses the volume serial number (VSN) and file ID for st_dev and st_ino. 
The OS allows the file ID to be 0 if the filesystem doesn't support file IDs. 
Also, it does not require or force the VSN to be a unique ID in the system, 
though if it's not 0 it's almost always a random 32-bit number for which the 
chance of collision is vanishingly small (notwithstanding a volume shadow copy, 
apparently). This means that (st_dev, st_ino) by itself is not sufficient to 
check whether two paths are the same file in Windows.

Proposal:

When comparing two file paths, if their (st_dev, st_ino) values differ, then 
they're not the same file. If their (st_dev, st_ino) values are the same, use 
the final NT paths from calling GetFinalPathNameByHandleW() with the flags 
VOLUME_NAME_NT | FILE_NAME_NORMALIZED. If only one of the paths supports 
FILE_NAME_NORMALIZED, then they're not the same file. If neither supports 
FILE_NAME_NORMALIZED, fall back on VOLUME_NAME_NT | FILE_NAME_OPENED. If either 
st_dev is 0 or st_ino is 0, the files are the same only if the final NT paths 
are the same. Else split out each device path. If the device paths are the 
same, then the paths are the same file. Otherwise they're different files.

We should probably special case the comparison of a multiple UNC provider path 
with a local volume path. For example r'\\localhost\C$\Windows' is the same as 
r'C:\Windows'. The corresponding NT paths are 
r'\Device\Mup\localhost\C$\Windows' and typically 
r'\Device\HarddiskVolume2\Windows'. The special case is that when one of the 
device paths is "\Device\Mup", the two device paths are not required to be the 
same. Of course, this is given that the (st_dev, st_ino) values are the same, 
and neither st_dev nor st_ino is zero.

That said, we would need to exclude volume shadow copies from the special case. 
I suppose we could just look for "VolumeShadowCopy" in the device name. Maybe 
we can do better. I've noticed that querying IOCTL_STORAGE_GET_DEVICE_NUMBER 
fails for a volume shadow copy, but that's probably going overboard.

--
nosy: +eryksun
stage:  -> needs patch
versions: +Python 3.10, Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29418] inspect.isroutine does not return True for some bound builtin methods


miss-islington  added the comment:


New changeset 562c13f5734d406b2283cfca673611f4b496fdc7 by Hakan Çelik in branch 
'main':
bpo-29418: Implement inspect.ismethodwrapper and fix inspect.isroutine for 
cases where methodwrapper is given (GH-19261)
https://github.com/python/cpython/commit/562c13f5734d406b2283cfca673611f4b496fdc7


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c

2022-02-16 Thread Pablo Galindo Salgado



Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46724] Odd Bytecode Generation in 3.10



Mark Shannon  added the comment:


New changeset d4e4ef107a9fea257981d7701f023177b704a44f by Mark Shannon in 
branch '3.10':
[3.10] bpo-46724: Use `JUMP_ABSOLUTE` for all backward jumps. (GH-31326) 
(GH-31354)
https://github.com/python/cpython/commit/d4e4ef107a9fea257981d7701f023177b704a44f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c



miss-islington  added the comment:


New changeset a657bff34945e40be24cd75d02560a93b7623cf5 by Miss Islington (bot) 
in branch '3.9':
bpo-46762: Fix an assert failure in f-strings where > or < is the last 
character if the f-string is missing a trailing right brace. (GH-31365)
https://github.com/python/cpython/commit/a657bff34945e40be24cd75d02560a93b7623cf5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46767] [Doc] sqlite3 Cursor.execute() return value is unspecified

2022-02-16 Thread Pierre Thierry



New submission from Pierre Thierry :

In the documentation of the sqlite3 module, the return value for 
Connection.execute() is told to be the Cursor that was implicitly created, but 
nothing is said about the return value/type when using Cursor.execute().

--
components: Library (Lib)
messages: 413327
nosy: kephas
priority: normal
severity: normal
status: open
title: [Doc] sqlite3 Cursor.execute() return value is unspecified
type: enhancement
versions: Python 3.10, Python 3.11, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c

2022-02-16 Thread Pablo Galindo Salgado



Pablo Galindo Salgado  added the comment:

Thanks for the quick fix, Eric!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c



miss-islington  added the comment:


New changeset 3d407b931156bc3ee5ae131de2f2babad831ccb2 by Miss Islington (bot) 
in branch '3.10':
bpo-46762: Fix an assert failure in f-strings where > or < is the last 
character if the f-string is missing a trailing right brace. (GH-31365)
https://github.com/python/cpython/commit/3d407b931156bc3ee5ae131de2f2babad831ccb2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.



Change by Dong-hee Na :


--
pull_requests: +29522
pull_request: https://github.com/python/cpython/pull/31372

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c



Change by miss-islington :


--
pull_requests: +29521
pull_request: https://github.com/python/cpython/pull/31371

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c



Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +29520
pull_request: https://github.com/python/cpython/pull/31370

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46762] assertion failure in f-string parsing Parser/string_parser.c

2022-02-16 Thread Eric V. Smith



Eric V. Smith  added the comment:


New changeset ffd9f8ff84ed53c956b16d027f7d2926ea631051 by Eric V. Smith in 
branch 'main':
bpo-46762: Fix an assert failure in f-strings where > or < is the last 
character if the f-string is missing a trailing right brace. (#31365)
https://github.com/python/cpython/commit/ffd9f8ff84ed53c956b16d027f7d2926ea631051


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46760] test_dis should test the dis module, not everything else



Change by Mark Shannon :


--
keywords: +patch
pull_requests: +29519
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31369

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46541] Replace _Py_IDENTIFIER() with statically initialized objects.