[issue1230540] sys.excepthook doesn't work in threads

[Christoph Reiter]

Christoph Reiter  added the comment:

> Let's say that in Python 3.8 threading.Thread now calls sys.execpthook() to 
> handle uncaught run() exception. All applications which override 
> sys.excepthook() on purpose will behave differently: start to log exceptions 
> from threads. But existing code is unlikely to be prepared to implement black 
> magic to check if we are a "thread" or the main thread, to decide if we 
> should display a thread name, and also the "black magic" to get the current 
> thread name.

Note that PyErr_Print() and PyErr_PrintEx() can be called in threads, and 
CPython itself uses it in some places which can be called in threads and I also 
use it in thread callbacks in C extensions I work on (PyGObject and pycairo for 
example). Nothing states currently that it's not allowed to call it in such 
cases :(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37018] put http.server on a diet

[Glenn Linderman]

New submission from Glenn Linderman :

The idea inspired by the email exchange below is basically in three parts:

1. investigate the various popular web server frameworks, to determine what 
parts of http.server they depend on. For example, bottle.py depends only on 
BaseHTTPRequestHandler and HTTPServer, which is less than half the code in 
http.server. Because of the deficiencies in the remaining parts, it seems 
unlikely that other frameworks use much more.

2. Trim http.server to those useful parts, removing the the rest from stdlib. 
Many of the "enhanced features" of http.server are such minimal enhancements 
that they are feature-poor and out-of-date with respect to current web server 
standards. The novice user is likely to be enticed into a swamp of missing 
capability when attempting to use them, as I was. It would take significant 
work to implement true CGI together with SSL on forking OSes; it took me 
significant work to implement true CGI together with SSL on Windows 
(non-forking). I gave up trying to do it on Linux, and switched to bottle.

3. Enhance what is left of http.server to support SSL and threading, so that 
the web frameworks that use http.server as a test server can at least offer 
those capabilities as well.  It isn't too hard to add those things for 
bottle.py, but it would be nicer if users didn't have to google for the blog 
posts that show how, and reimplement it (most of the blog posts are somewhat 
dated).

On 5/22/2019 4:09 AM, Christian Heimes wrote:
> On 22/05/2019 01.11, Glenn Linderman wrote:
>> On 5/21/2019 2:00 PM, Nathaniel Smith wrote:
>>> On Tue, May 21, 2019 at 10:43 AM Glenn Linderman  
>>> wrote:
 After maintaining my own version of http.server to fix or workaround some 
 of its deficiencies for some years, I discovered bottle.py. It has far 
 more capability, is far better documented, and is just as quick to deploy. 
 While I haven't yet converted all past projects to use bottle.py, it will 
 likely happen in time, unless something even simpler to use is discovered, 
 although I can hardly imagine that happening.
>>> bottle.py uses http.server for its local development mode (the one you
>>> see in their quickstart example at the top of their README). Same with
>>> flask, django, and probably a bunch of other frameworks. It's *very*
>>> widely used.
>>>
>>> -n
>>>
>> The source for bottle.py version 0.13-dev has an import for http.client, but 
>> not http.server. I hadn't tracked down every indirect dependency in the 
>> bottle.py source code, but it seems that if one uses the "default server" 
>> for bottle, that it is "wsgiref", imported from wsgiref.simple_server, and 
>> that in turn does import BaseHTTPRequestHandler and HTTPServer from 
>> http.server.
>>
>> It is the higher-level code in http.server that has significant deficiencies 
>> that have caused me problems over the years... a "SimpleHTTPRequestHandler" 
>> that is so simple it doesn't do POST, PUT or PASTE, a 
>> "CGIHTTPRequestHandler" that only implements part of the CGI protocol, only 
>> CGI support in POST, no support for PUT or PASTE, and no support for https, 
>> and not much bug fix activity in those areas.
>>
>> Maybe http.server should be split into the "basic parts" (used by bottle.py, 
>> and other frameworks), and the "higher-level parts", which could then be 
>> discarded by this PEP! At this point, though, I'd have to agree that the 
>> whole should not be discarded. Thanks for making me dig deeper.
>
> The idea has merrit. However I feel its out of scope for the PEP [594]. The 
> http.server module and socketserver module are still widely used for debug 
> and toy examples.
>
> Could you please open a bug to track your proposal? We may pursue it in a 
> couple of years from now.

--
messages: 343275
nosy: v+python
priority: normal
severity: normal
status: open
title: put http.server on a diet

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[Inada Naoki]

Change by Inada Naoki :


--
pull_requests:  -7043

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33164] Blake 2 module update

[Inada Naoki]

New submission from Inada Naoki :


New changeset 51aa35e9e17eef60d04add9619fe2a7eb938358c by Inada Naoki (David 
Carlier) in branch 'master':
bpo-33164: update blake2 implementation (GH-6286)
https://github.com/python/cpython/commit/51aa35e9e17eef60d04add9619fe2a7eb938358c


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] Implementation of the PEP 587

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 20e1e2582e5e69e43af88ff58699c8883d146acb by Victor Stinner in 
branch 'master':
bpo-36763: Fix _PyPreConfig_InitCompatConfig() utf8_mode (GH-13518)
https://github.com/python/cpython/commit/20e1e2582e5e69e43af88ff58699c8883d146acb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21315] email._header_value_parser does not recognise in-line encoding changes

[Abhilash Raj]

Abhilash Raj  added the comment:

I have made the requested changes on PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35805] email package folds msg-id identifiers using RFC2047 encoded words where it must not

[Abhilash Raj]

Abhilash Raj  added the comment:

I have made the requested changes on PR.

David, can you please review again?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18748] io.IOBase destructor silence I/O error on close() by default

[STINNER Victor]

STINNER Victor  added the comment:


New changeset bc2aa816620c5e02ad8e94d8514b7e8f3f551ca1 by Victor Stinner in 
branch 'master':
bpo-18748: _pyio.IOBase emits unraisable exception (GH-13512)
https://github.com/python/cpython/commit/bc2aa816620c5e02ad8e94d8514b7e8f3f551ca1


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27737] email.header.Header.encode() crashes with IndexError on spaces only value

[R. David Murray]

R. David Murray  added the comment:


New changeset 0416d6f05a96e0f1b3751aa97abfffe6d3323976 by R. David Murray (Miss 
Islington (bot)) in branch '3.7':
bpo-27737: Allow whitespace only headers encoding (GH-13478) (#13517)
https://github.com/python/cpython/commit/0416d6f05a96e0f1b3751aa97abfffe6d3323976


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36520] Email header folded incorrectly

[R. David Murray]

R. David Murray  added the comment:

Nevermind, I was testing with the wrong version of python.  This bug was 
introduced somewhere after 3.4 :(

>>> from email.message import EmailMessage
>>> m = EmailMessage()
>>> m['Subject'] = 'Hello Wörld! Hello Wörld! Hello Wörld! Hello Wörld!Hello 
>>> Wörld!'
>>> bytes(m)
b'Subject: Hello =?utf-8?q?W=C3=B6rld!_Hello_W=C3=B6rld!_Hello_W=C3=B6rld!?=\n 
Hello =?utf-8?=?utf-8?q?q=3FW=3DC3=3DB6rld!Hello=3F=3D_W=C3=B6rld!?=\n\n'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36520] Email header folded incorrectly

[R. David Murray]

R. David Murray  added the comment:

Can you demonstrate the problem with an actual email object?  
header_store_parse is not meant to be called directly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36721] Add pkg-config python-3.8-embed and --embed to python3.8-config

[STINNER Victor]

STINNER Victor  added the comment:

Even if I'm not confident in my change (add --embed option), I chose to merge 
it anyway since at least "waf" build system is broken by my other changes (no 
longer link C extensions to libpython). I would like to get this change into 
Python 3.8 beta1 to attempt to fix most applications embedding Python.

Anyway, if something goes wrong, we still have plenty of time to decide what to 
do before 3.8.0 final, scheduled for 2019-10-21: 
https://www.python.org/dev/peps/pep-0569/

--

Since I merged my change, I reset the priority from Release Blocker to normal.

--
priority: release blocker -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36721] Add pkg-config python-3.8-embed and --embed to python3.8-config

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0a8e57248b913851640c64375600f05157c997df by Victor Stinner in 
branch 'master':
bpo-36721: Add --embed option to python-config (GH-13500)
https://github.com/python/cpython/commit/0a8e57248b913851640c64375600f05157c997df


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] Implementation of the PEP 587

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13435

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27737] email.header.Header.encode() crashes with IndexError on spaces only value

[R. David Murray]

R. David Murray  added the comment:

Thanks.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.7, Python 3.8 -Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37000] _randbelow_with_getrandbits function inefficient with powers of two

[Tim Peters]

Tim Peters  added the comment:

I believe the thrust of Mark's suggestion was that it would allow using `k = 
(n-1).bit_length()` even when n == 1, without special-casing n == 1.  But you'd 
still be adding a new "subtract 1" operation, and would still change results in 
some cases.

That said, given that `(0).bit_length() == 0`, it's a bit surprising all on its 
own that `getrandbits(0)` raises an exception.

In any case, I'd leave _randbelow_with_getrandbits alone.  Conauming "extra" 
bits is generally a red herring, since the underlying `getrandbits()` consumes 
32 bits at a time from the Twister.  That is, we're _typically_ "wasting" more 
than a dozen bits regardless already (e.g., getrandbits(2), getrandbits(17), 
and getrandbits(29) all consume 32 bits).

It's unfortunate that _randbelow_with_getrandbits(power_of_2) may invoke 
getrandbits() more than once.  But there's also a bright side:  because there's 
always the possibility that _randbelow_with_getrandbits() may invoke 
getrandbits() more than once, we can't guess how many times getrandbits() _was_ 
called.  So, in turn, we can't know how much of the Twister's state space was 
consumed.  Which, in turn, makes it much harder to deduce the Twister's' 
internal state from the visible outputs (but this can be done with certainty 
from a long enough string of, say, random.choice([0, 1]) outputs if we knew 
getrandbits was called exactly once for each, despite that we're only seeing 1 
bit of each 32-bit Twister output).

That last point shouldn't drive anything, but it is kinda pleasant that people 
inappropriately using the Twister in contexts where keeping secrets is 
important are partially protected by under-the-covers accept/reject methods.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27737] email.header.Header.encode() crashes with IndexError on spaces only value

[miss-islington]

Change by miss-islington :


--
pull_requests: +13434

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27737] email.header.Header.encode() crashes with IndexError on spaces only value

[R. David Murray]

R. David Murray  added the comment:


New changeset ef5bb25e2d6147cd44be9c9b166525fb30485be0 by R. David Murray 
(Batuhan Taşkaya) in branch 'master':
bpo-27737: Allow whitespace only headers encoding (#13478)
https://github.com/python/cpython/commit/ef5bb25e2d6147cd44be9c9b166525fb30485be0


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

About threading.excepthook() API, maybe we should not reproduce 
sys.excepthook() API but instead reuse something closer to sys.unraisablehook() 
API: use a single parameter which has attributes. It would allow to pass more 
parameters as new attributes in the future, maybe some new "optional" 
parameters (None by default).

For example, we can imagine calling threading.excepthook() to handle 
threading.excepthook() failure. We would need an argument to prevent an infine 
loop :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

I dislike PR 8610: threading.Thread doesn't call sys.excepthook to handle run() 
exception by default, it only calls sys.excepthook if it's overridden. 
Moreover, when sys.excepthook is called, the hook doesn't get access to the 
thread object :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37017] Use LOAD_METHOD optimization in CallMethod C API functions

[Michael J. Sullivan]

Change by Michael J. Sullivan :


--
nosy: +brett.cannon, serhiy.storchaka, vstinner, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37017] Use LOAD_METHOD optimization in CallMethod C API functions

[Michael J. Sullivan]

Change by Michael J. Sullivan :


--
keywords: +patch
pull_requests: +13433
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37017] Use LOAD_METHOD optimization in CallMethod C API functions

[Michael J. Sullivan]

New submission from Michael J. Sullivan :

The different varieties of PyObject_CallMethod* routines all operate by doing a 
PyObject_GetAttr to fetch an object to call. It seems likely to be worthwhile 
to take advantage of the LOAD_METHOD optimization that avoids creating a bound 
method object when calling a method.

--
components: Extension Modules
messages: 343259
nosy: msullivan
priority: normal
severity: normal
status: open
title: Use LOAD_METHOD optimization in CallMethod C API functions
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35091] Objects/listobject.c: gallop functions rely on signed integer overflow

[Cheryl Sabella]

Change by Cheryl Sabella :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

I wrote PR 13515 which adds threading.excepthook(). I chose to call 
threading.excepthook() even when run() raises SystemExit. In this case, 
threading.excepthook() simply does nothing. The idea is to really give the full 
control when threading.excepthook() is overriden. For example, log a warning 
when run() raises SystemExit. By the way, is it really a good idea to call 
sys.exit() from a thread? It sounds like a bug that should be reported, and not 
silently ignored, no?

Differences between sys.excepthook() and threading.excepthook():

* API: sys.excepthook(exctype, value, traceback, /) vs 
threading.excepthook(exc_type, exc_value, exc_tb, thread, /) -- addition thread 
parameter to display the name of the thread which raises an exception

* For SystemExit corner case, sys.excepthook() displays the exception, whereas 
threading.excepthook() silently ignores it

* When sys.stderr is None, sys.excepthook() does nothing, whereas 
threading.excepthook() tries harder: use its own copy of sys.stderr (saved when 
the thread has been created) from thread._stderr.

Thread._stderr was added by bpo-754449:

commit cc4e935ea593cede10cb1316e3faeabd708abca7
Author: Brett Cannon 
Date:   Sat Jul 3 03:52:35 2004 +

threading.Thread objects will now print a traceback for an exception raised
during interpreter shutdown instead of masking it with another traceback 
about
accessing a NoneType when trying to print the exception out in the first 
place.

Closes bug #754449 (using patch #954922).

Note: When sys.stderr is None, threading.excepthook() avoids the traceback 
module and renders the exception itself. Maybe threading.excepthook() should be 
reimplemented in C to make it even more reliable and more correct, especially 
during Python shutdown. Only daemon threads are impacted: Python finalization 
(Py_Finalize() C function) starts by calling threading._shutdown() which joins 
all non-daemon threads.


IMHO the threading.Thread semantics is too different than sys.excepthook() to 
reuse sys.excepthook() to handle threading.Thread.run() exception.

Another explanation is that sadly sys.excepthook() API uses exactly 3 
positional-only arguments, and so the API cannot be easily extended to get a 
thread argument. When I designed sys.unraisablehook(), I chose to pass only one 
argument which has attributes, to prevent this issue.

I'm not comfortable to attempt to modify sys.excepthook() to make it behave 
differently if it's called from the main thread or from a different thread. It 
would have to call threading.current_thread().name to get the name of the 
current thread.


Let's say that in Python 3.8 threading.Thread now calls sys.execpthook() to 
handle uncaught run() exception. All applications which override 
sys.excepthook() on purpose will behave differently: start to log exceptions 
from threads. But existing code is unlikely to be prepared to implement black 
magic to check if we are a "thread" or the main thread, to decide if we should 
display a thread name, and also the "black magic" to get the current thread 
name.


One of my concern of reusing sys.excepthook to display threading exceptions is 
that adding more code to handle threads is a risk of raising a new exception 
while logging a threading exception :-(

IMHO threading.excepthook() is safer since it already has access to the thread.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35091] Objects/listobject.c: gallop functions rely on signed integer overflow

[miss-islington]

miss-islington  added the comment:


New changeset 367fe5757a707c4e3602dee807a9315199ed0b5c by Miss Islington (bot) 
in branch '3.7':
bpo-35091: Objects/listobject.c: Replace overflow checks in gallop fu… 
(GH-10202)
https://github.com/python/cpython/commit/367fe5757a707c4e3602dee807a9315199ed0b5c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13432

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35091] Objects/listobject.c: gallop functions rely on signed integer overflow

[miss-islington]

miss-islington  added the comment:


New changeset 6bc5917903b722bdd0e5d3020949f26fec5dfe9a by Miss Islington (bot) 
(Alexey Izbyshev) in branch 'master':
bpo-35091: Objects/listobject.c: Replace overflow checks in gallop fu… 
(GH-10202)
https://github.com/python/cpython/commit/6bc5917903b722bdd0e5d3020949f26fec5dfe9a


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35091] Objects/listobject.c: gallop functions rely on signed integer overflow

[miss-islington]

Change by miss-islington :


--
pull_requests: +13431

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

STINNER Victor  added the comment:


New changeset df22c03b93ea4620fdf4a0b3cbbbfa7c645af783 by Victor Stinner in 
branch 'master':
bpo-36829: PyErr_WriteUnraisable() normalizes exception (GH-13507)
https://github.com/python/cpython/commit/df22c03b93ea4620fdf4a0b3cbbbfa7c645af783


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] Implementation of the PEP 587

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 5edcf263581c70f6a6c2206db679e51e9418bb38 by Victor Stinner in 
branch 'master':
bpo-36763: Rename private Python initialization functions (GH-13511)
https://github.com/python/cpython/commit/5edcf263581c70f6a6c2206db679e51e9418bb38


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-1230540: "sys.excepthook doesn't work in threads".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue1230540] sys.excepthook doesn't work in threads

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-36829: I just added sys.unraisablehook().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13430

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36941] Windows build changes for Windows ARM64

[Steve Dower]

Steve Dower  added the comment:


New changeset cfb241bd29a94fd825a317a78322e3cdba0e75a7 by Steve Dower (Paul 
Monson) in branch 'master':
bpo-36941: Project file fixups for Windows ARM64 (GH-13477)
https://github.com/python/cpython/commit/cfb241bd29a94fd825a317a78322e3cdba0e75a7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

STINNER Victor  added the comment:

I wrote PR 13512 to use support.catch_unraisable_exception() in 
test_io.test_error_through_destructor(). But this PR is associated to bpo-18748 
since the main change is related to the io module, not sys.unraisablehook ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18748] io.IOBase destructor silence I/O error on close() by default

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13429

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36918] ValueError warning in test_urllib due to io.IOBase destructor

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13428

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33110] Adding a done callback to a concurrent.futures Future once it has already completed, may raise an exception, contrary to docs

[miss-islington]

miss-islington  added the comment:


New changeset b73c21c0be7b42de6a88d67408249c8ec46e28f7 by Miss Islington (bot) 
in branch '3.7':
bpo-33110: Catch errors raised when running add_done_callback on already 
completed futures (GH-13141)
https://github.com/python/cpython/commit/b73c21c0be7b42de6a88d67408249c8ec46e28f7


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] Implementation of the PEP 587

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13427

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36763] Implementation of the PEP 587

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 022be02dcfdfd9011415804bb4553a33fa7ec8f3 by Victor Stinner in 
branch 'master':
bpo-36763: Add _PyPreConfig._config_init (GH-13481)
https://github.com/python/cpython/commit/022be02dcfdfd9011415804bb4553a33fa7ec8f3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

STINNER Victor  added the comment:

In PR 13490, Thomas Grainger proposed a cool context manager:

@contextlib.contextmanager
def throw_unraisable_exceptions():
unraisable = None
old_hook = sys.unraisablehook

def hook(exc):
nonlocal unraisable
unraisable = exc

sys.unraisablehook = hook
try:
yield
if unraisable is not None:
raise unraisable
finally:
unraisable = None
sys.unraisablehook = old_hook

It allows to raise an unraisable exception :-D Example:

try:
with support.throw_unraisable_exceptions():
...
except Exception as e:
... # the exception is now here

I don't need such context manager right now, but I like the fact that it 
becomes possible to write such context manager :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37016] Python embedded in C++ cannot open a file

[Soumya Mohanty]

New submission from Soumya Mohanty :

Hello,
I am trying to open a pickled file and load it in my python file.

This python file will be called from a c++ program. Please find the C++ program 
attached.  

Py_Initialize and Py_Finalize are being done in a separate file called 
pyhelper.hpp

Python code :

def test():
print("In function test of pyemb.py file \n")
import pickle
with open('filepath', 'rb') as f_in:
C = pickle.load(f_in)

I am getting the following error and cant find any way to fix it:

Exception ignored in: 
Traceback (most recent call last):
  File "C:\Anaconda3\envs\Deep_Learning\lib\threading.py", line 1289, in 
_shutdown
assert tlock.locked()
SystemError:  returned a result with an error set

--
files: PYTHON_PROJ.cpp
messages: 343246
nosy: mohantys
priority: normal
severity: normal
status: open
title: Python embedded in C++ cannot open a file
versions: Python 3.6
Added file: https://bugs.python.org/file48350/PYTHON_PROJ.cpp

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e4d300e07c33a9a77549c62d8687d8fe130c53d5 by Victor Stinner in 
branch 'master':
bpo-36829: Add test.support.catch_unraisable_exception() (GH-13490)
https://github.com/python/cpython/commit/e4d300e07c33a9a77549c62d8687d8fe130c53d5


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24882] ThreadPoolExecutor doesn't reuse threads until #threads == max_workers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Thank you for your contribution iunknwn!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33110] Adding a done callback to a concurrent.futures Future once it has already completed, may raise an exception, contrary to docs

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
versions: +Python 3.7, Python 3.8 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13426

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33110] Adding a done callback to a concurrent.futures Future once it has already completed, may raise an exception, contrary to docs

[miss-islington]

Change by miss-islington :


--
pull_requests: +13425

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24882] ThreadPoolExecutor doesn't reuse threads until #threads == max_workers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 904e34d4e6b6007986dcc585d5c553ee8ae06f95 by Antoine Pitrou (Sean) 
in branch 'master':
bpo-24882: Let ThreadPoolExecutor reuse idle threads before creating new thread 
(#6375)
https://github.com/python/cpython/commit/904e34d4e6b6007986dcc585d5c553ee8ae06f95


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33110] Adding a done callback to a concurrent.futures Future once it has already completed, may raise an exception, contrary to docs

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 2a3a2ece502c05ea33c95dd0db497189e0354bfd by Antoine Pitrou (Sam 
Martin) in branch 'master':
bpo-33110: Catch errors raised when running add_done_callback on already 
completed futures (GH-13141)
https://github.com/python/cpython/commit/2a3a2ece502c05ea33c95dd0db497189e0354bfd


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33110] Adding a done callback to a concurrent.futures Future once it has already completed, may raise an exception, contrary to docs

[miss-islington]

Change by miss-islington :


--
pull_requests: +13424

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 942c31dffbe886ff02e25a319cc3891220b8c641 by Victor Stinner in 
branch '2.7':
bpo-35907: Complete test_urllib.test_local_file_open() (GH-13506)
https://github.com/python/cpython/commit/942c31dffbe886ff02e25a319cc3891220b8c641


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 34bab215596671d0dec2066ae7d7450cd73f638b by Victor Stinner in 
branch '3.7':
bpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme (GH-13474) 
(GH-13505)
https://github.com/python/cpython/commit/34bab215596671d0dec2066ae7d7450cd73f638b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36878] ast.parse with type_comments=True should allow extra text after # type: ignore

[Michael J. Sullivan]

Michael J. Sullivan  added the comment:

I think this is done!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Matthias Bussonnier]

Matthias Bussonnier  added the comment:

As a reference, PR from Yuri for an asyncREPL  `python -m asyncio` which I 
believe he want in 3.8:

https://github.com/python/cpython/pull/13472

I'm also likely to align IPython behavior on whatever core python decides.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36878] ast.parse with type_comments=True should allow extra text after # type: ignore

[Guido van Rossum]

Guido van Rossum  added the comment:

Thanks!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13423

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36878] ast.parse with type_comments=True should allow extra text after # type: ignore

[miss-islington]

miss-islington  added the comment:


New changeset d8a82e2897b735e2b7e9e086f1d709365a2ad72c by Miss Islington (bot) 
(Michael J. Sullivan) in branch 'master':
bpo-36878: Only allow text after `# type: ignore` if first character ASCII 
(GH-13504)
https://github.com/python/cpython/commit/d8a82e2897b735e2b7e9e086f1d709365a2ad72c


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37000] _randbelow_with_getrandbits function inefficient with powers of two

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Some quick notes:

* In issue 33144, we achieved a significant speed-up for 
_randbelow_with_getrandbits() by removing a single test.  The code for that 
method is thin and almost any additional logic will slow it down.

* The attached PR (now closed) causes a performance regression.  Shuffling a 
thousand element list regressed from 505 usec per loop to 576 usec per loop.

* We only promise that the output of random() will be reproducible across 
versions; however, we should have an aversion to changing the output of the 
other methods unless it is really necessary (because it may change the result 
of simulations or random selections which will cause some consternation for 
some end-users).  For seed(8675309), the result of "[randrange(1024) for i in 
range(10)]" changes under the PR from [823, 438, 575, 465, 718, 186, 25, 1015, 
654, 988] to [411, 219, 522, 961, 679, 516, 881, 919, 287, 882].  This is 
allowed but not desireable.

When I get a chance, I'll take a closer look at Mark's suggestion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13422

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +13421

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Agree with Nathaniel.
There is no need to rush now.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35907] [security][CVE-2019-9948] Unnecessary URL scheme exists to allow local_file:// reading file in urllib

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 0c2b6a3943aa7b022e8eb4bfd9bffcddebf9a587 by Victor Stinner in 
branch 'master':
bpo-35907, CVE-2019-9948: urllib rejects local_file:// scheme (GH-13474)
https://github.com/python/cpython/commit/0c2b6a3943aa7b022e8eb4bfd9bffcddebf9a587


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37006] Add top level await statement support for doctest

[Nathaniel Smith]

Nathaniel Smith  added the comment:

As far as things like run/run_until_complete/etc go, I think doctests should 
have the same semantics as async REPLs, whatever those end up being.

Given that we don't actually have mature async REPLs, that the core feature to 
enable them only landed a few days ago, and the 3.8 freeze is almost here, I 
think we should defer the doctest discussion for 3.9. Hopefully by then, things 
will have settled down more in async REPL land, and it'll be more obvious what 
doctest should do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31862] Port the standard library to PEP 489 multiphase initialization

[Sergey Fedoseev]

Change by Sergey Fedoseev :


--
pull_requests: +13420

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34616] implement "Async exec"

[miss-islington]

miss-islington  added the comment:


New changeset 2ddbd21aec7f0e2f237a1073d3e0b313e673413f by Miss Islington (bot) 
(Matthias Bussonnier) in branch 'master':
bpo-34616: Document top level async in whatsnew/3.8. (GH-13484)
https://github.com/python/cpython/commit/2ddbd21aec7f0e2f237a1073d3e0b313e673413f


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37015] Fix asyncio mock warnings

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I guess cause is at [0] . When there is no new value specified and with spec 
being None if mock.patch is used on an async object (original = 
_accept_connection2). Here original is an async object then AsyncMock is 
returned. 

Changing this causes test failures where patching AsyncClass.async_method now 
expects an AsyncMock but as per older behavior it returns MagicMock. I think 
it's a behavior that needs to be discussed as it differs from 3.7 and not sure 
if AsyncMock would always be awaited as seen in test_accept_connection_multiple.

[0] 
https://github.com/python/cpython/blob/b121f63155d8e3c7c42ab6122e36eaf7f5e9f7f5/Lib/unittest/mock.py#L1313

if spec is None and _is_async_obj(original):
Klass = AsyncMock
else:
Klass = MagicMock

--
nosy: +lisroach, mariocj89

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36878] ast.parse with type_comments=True should allow extra text after # type: ignore

[Michael J. Sullivan]

Change by Michael J. Sullivan :


--
pull_requests: +13419

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37015] Fix asyncio mock warnings

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

patching _accept_connection2 attribute on loop object seems to return an 
AsyncMock.

➜  cpython git:(master) ✗ cat ../backups/bpo37015.py
import asyncio
from unittest.mock import patch
with patch.object(asyncio.get_event_loop(), '_accept_connection2') as f:
print(f)
f()
➜  cpython git:(master) ✗ ./python.exe ../backups/bpo37015.py

../backups/bpo37015.py:5: RuntimeWarning: coroutine 'AsyncMockMixin._mock_call' 
was never awaited
  f()
RuntimeWarning: Enable tracemalloc to get the object allocation traceback


Relevant test

def test_accept_connection_multiple(self):
sock = mock.Mock()
sock.accept.return_value = (mock.Mock(), mock.Mock())
backlog = 1
# Mock the coroutine generation for a connection to prevent
# warnings related to un-awaited coroutines.
mock_obj = mock.patch.object
with mock_obj(self.loop, '_accept_connection2') as accept2_mock:
print(f"{accept2_mock=}")
accept2_mock.return_value = None
with mock_obj(self.loop, 'create_task') as task_mock:
task_mock.return_value = None
self.loop._accept_connection(
mock.Mock(), sock, backlog=backlog)
self.assertEqual(sock.accept.call_count, backlog)

When I specify new value which defaults to DEFAULT as Mock() then there is no 
AsyncMock. Same can be done in test and the warnings go away. My suspicion is 
that if there is a loop object with _accept_connection2 earlier in Python 3.7 
 is returned by patch.object but now it 
returns an  instead

# use explicit mock

import asyncio
from unittest.mock import patch, Mock

with patch.object(asyncio.get_event_loop(), '_accept_connection2', Mock()) as f:
print(f)
f()

➜  cpython git:(master) ✗ ./python.exe ../backups/bpo37015.py


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37015] Fix asyncio mock warnings

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Not sure if it helps. Set backlog in the below test as 1. Add a print statement 
print(type(_mock_call)) at [0] . I could see some of _mock_call to be AsyncMock 
. So instead of calling return _mock_self._mock_call(*args, **kwargs) directly 
if I call them only with they are not instance of AsyncMock (not 
isinstance(_mock_self, AsyncMock)) then the warning goes away. 

Guess somewhere an AsyncMock is created as I can see _accept_connection2. When 
the backlog value is set to 100 it shows lot of warnings.

$ ./python.exe -Werror -X tracemalloc -m unittest -vv 
test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests.test_accept_connection_multiple


test_accept_connection_multiple 
(test.test_asyncio.test_selector_events.BaseSelectorEventLoopTests) ... 






ok

--
Ran 1 test in 0.123s

OK
Exception ignored in: 
Traceback (most recent call last):
  File "/Users/karthikeyansingaravelan/stuff/python/cpython/Lib/warnings.py", 
line 510, in _warn_unawaited_coroutine
warn(msg, category=RuntimeWarning, stacklevel=2, source=coro)
RuntimeWarning: coroutine 'AsyncMockMixin._mock_call' was never awaited

[0] 
https://github.com/python/cpython/blob/b121f63155d8e3c7c42ab6122e36eaf7f5e9f7f5/Lib/unittest/mock.py#L991

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37003] ast unparse does not support f-string new debug format.

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Notice that test_tools will fail if  f'{x=}' becomes f'x={x!r}'

I arrived at the wrong conclusion as Matthias points out.

>>> import ast
>>> ast.dump(compile("f'{x=}'","","exec",ast.PyCF_ONLY_AST))
"Module(body=[Expr(value=JoinedStr(values=[FormattedValue(value=Name(id='x', 
ctx=Load()), conversion=114, format_spec=None, expr_text='x=')]))], 
type_ignores=[])"
>>> ast.dump(compile("f'{x!r}'","","exec",ast.PyCF_ONLY_AST))
"Module(body=[Expr(value=JoinedStr(values=[FormattedValue(value=Name(id='x', 
ctx=Load()), conversion=114, format_spec=None, expr_text=None)]))], 
type_ignores=[])"

if expr_text is removed those strings will be the same, so we will be ok. Sorry 
for the confusion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37003] ast unparse does not support f-string new debug format.

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Actually, it checks if the dump is the same:

 class ASTTestCase(unittest.TestCase):
def assertASTEqual(self, ast1, ast2):
self.assertEqual(ast.dump(ast1), ast.dump(ast2))

def check_roundtrip(self, code1, filename="internal"):
ast1 = compile(code1, filename, "exec", ast.PyCF_ONLY_AST)
unparse_buffer = io.StringIO()
unparse.Unparser(ast1, unparse_buffer)
code2 = unparse_buffer.getvalue()
ast2 = compile(code2, filename, "exec", ast.PyCF_ONLY_AST)
self.assertASTEqual(ast1, ast2)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[Michael Felt]

Michael Felt  added the comment:

On 22/05/2019 18:08, STINNER Victor wrote:
> STINNER Victor  added the comment:
>
> Michael Felt: it's annoying when you ignore Antoine's comment and my comment.
> * https://github.com/python/cpython/pull/13463#issuecomment-494797084
> * https://bugs.python.org/issue36084#msg343159
>
> The AIX case is very special and required a separated issue. Please open a 
> separated if you want to discuss/implement get_native_id() on AIX.

My apologies. I was not ignoring anyone. I am sorry you had that impression.

I had already taken it as a given that it would not be in this PR (re:
https://bugs.python.org/issue36084#msg343159)

And, I had expressed my hope - it would not be too complex in
https://bugs.python.org/issue36084#msg343168. Which also made me realize
that an 'issue' around the "define" described in the "pthread"
documentation (so in hindsight, not applicable to an implementation of
"get_native_id()".

And, while I am sorry you feel I have ignored you - it is hardly the
case. Everyone's comments have given me a reason to look further. And it
grieves me that my intentions are misunderstood.

Thank you for your honesty!

>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37015] Fix asyncio mock warnings

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[Jake Tesler]

Jake Tesler  added the comment:

Victor – the return value of _start_new_thread is the the `ident` parameter, 
and its not the same as the native id.

See here: https://github.com/python/cpython/pull/11993#issuecomment-491544908

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37015] Fix asyncio mock warnings

[Andrew Svetlov]

Change by Andrew Svetlov :


--
title: Fix asyncio mock wranings -> Fix asyncio mock warnings

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37015] Fix asyncio mock wranings

[Andrew Svetlov]

New submission from Andrew Svetlov :

After merging https://github.com/python/cpython/pull/9296 asyncio test suite 
prints a lot of warnings like

Exception ignored in: 
Traceback (most recent call last):
  File "/home/andrew/projects/cpython/Lib/warnings.py", line 510, in 
_warn_unawaited_coroutine
warn(msg, category=RuntimeWarning, stacklevel=2, source=coro)
RuntimeWarning: coroutine 'AsyncMockMixin._mock_call' was never awaited

I believe this is not a sign of AsyncMock problem but asyncio tests should be 
fixed.
Need more investigations though.

--
components: Tests
messages: 343223
nosy: asvetlov
priority: normal
severity: normal
status: open
title: Fix asyncio mock wranings
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37003] ast unparse does not support f-string new debug format.

[Matthias Bussonnier]

Matthias Bussonnier  added the comment:

I thought it was comparing the AST of the file to the AST of the unparsed
AST.

So it's actually checking if parse(unparse(x)) is indempotent and not
wether parse(unparse(x)) is indempotent.

So x={x!r} should be fine.

On Wed, May 22, 2019, 09:22 Pablo Galindo Salgado 
wrote:

>
> Pablo Galindo Salgado  added the comment:
>
> Notice that test_tools will fail if  f'{x=}' becomes f'x={x!r}' when
> unparsed as it compares the text of the file and the text of the roundtrip
> of the ast of the file
>
> --
> nosy: +pablogsal
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33482] codecs.StreamRecoder.writelines is broken

[Guido van Rossum]

Change by Guido van Rossum :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37014] fileinput module should document that openhook and mode are ignored when reading from stdin

[Grant Wu]

New submission from Grant Wu :

https://github.com/python/cpython/blob/master/Lib/fileinput.py#L326 shows that 
the openhook and mode are ignored when reading from stdin.

Since part of fileinput's functionality is to abstract over whether one is 
reading from stdin or over a file, I think this abstraction leak should be 
documented.  One common use case where this might break is when attempting to 
set the file encoding using the included fileinput.hook_encoded functionality.

--
assignee: docs@python
components: Documentation
messages: 343221
nosy: Grant Wu2, docs@python
priority: normal
severity: normal
status: open
title: fileinput module should document that openhook and mode are ignored when 
reading from stdin
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37011] pdb: restore original tracing function instead of sys.settrace(None)

[daniel hahler]

daniel hahler  added the comment:

Looks great, thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36905] test_typing.GetTypeHintTests.test_get_type_hints_modules_forwardref unexpected success while running whole test suite sequentially

[Matej Cepl]

Change by Matej Cepl :


--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27987] obmalloc's 8-byte alignment causes undefined behavior

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Neil, I don't see the point of having this discussion here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27987] obmalloc's 8-byte alignment causes undefined behavior

[Neil Schemenauer]

Neil Schemenauer  added the comment:

We now have a concrete use case. ;-) 
 
My idea was that we can introduce a new, CPython internal API that
aligns on 8-byte boundaries (or takes alignment as a parameter).  The
API would be a stop-gap measure. We can use the API to reduce
the overhead for specific types. 
 
E.g. for non-subclasses of float, we know the PyObject structure does
not need 16-byte alignment. We don't need a version of "alignof" to know
this. Inside floatobject.c, we could call the new objmalloc API that
gives new memory with 8-byte alignment. That would save the 33%
overhead. 
 
E.g. in PyFloat_FromDouble, rather than: 
 
 PyObject_MALLOC(sizeof(PyFloatObject))  
 
we could call something like: 
 
 _PyObject_MALLOC_ALIGNED(sizeof(PyFloatObject), 8)  
 
This internal API would not be a permanent solution. Having to manually
fix each place that PyObjects are allocated and hard-coding the required
alignment is not the best solution. We can only fix specific types and
extension modules would always get the 16-byte alignment. Still, by
tweaking some of the most common types, we avoid much of the overhead
for the alignment change, at least for the average Python program. 
 
In the long term, we would need a better solution. E.g. an API that can
take alignment requirements as a parameter. Or, a solution I like
better, have types use PyObject_New().  Then, add an alignment
specifier to type object (e.g. tp_align to go along with tp_basicsize).
Then there does not have to be a new public API that takes alignment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35760] test_asyncio: test_async_gen_asyncio_gc_aclose_09() race condition

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

This seems to occur rather commonly now such that it fails and then passes on 
verbose run. Example : 
https://dev.azure.com/Python/cpython/_build/results?buildId=43325=logs=c83831cd-3752-5cc7-2f01-8276919eb334=5a421c4a-0933-53d5-26b9-04b36ad165eb=536

Victor, do you agree on increasing the timeout for sleep to 1 for this test?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37013] Fatal Python error in socket.if_indextoname()

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +13418
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33482] codecs.StreamRecoder.writelines is broken

[miss-islington]

miss-islington  added the comment:


New changeset 81c5ec9e417aebfe92945a05771006e4241f4e08 by Miss Islington (bot) 
(Jelle Zijlstra) in branch '3.7':
[3.7] bpo-33482: fix codecs.StreamRecoder.writelines (GH-6779) (GH-13502)
https://github.com/python/cpython/commit/81c5ec9e417aebfe92945a05771006e4241f4e08


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37013] Fatal Python error in socket.if_indextoname()

[Zackery Spytz]

New submission from Zackery Spytz :

Python 3.8.0a4+ (heads/master:ef9d9b6312, May 22 2019, 08:35:25) 
[GCC 9.0.1 20190402 (experimental) [trunk revision 270074]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.if_indextoname(2**64 - 1)
Fatal Python error: a function returned NULL without setting an error
SystemError:  returned NULL without setting 
an error

Current thread 0x7f29d708d140 (most recent call first):
  File "", line 1 in 
Aborted (core dumped)

This is because socket.if_indextoname() does not use PyErr_Occurred() when 
checking PyLong_AsUnsignedLong() for failure.

--
components: Extension Modules
messages: 343215
nosy: ZackerySpytz, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Fatal Python error in socket.if_indextoname()
type: crash
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses

[twisteroid ambassador]

twisteroid ambassador  added the comment:

With regards to the failing test, it looks like the test basically boils down 
to testing whether loop.getaddrinfo('fe80::1%1', 80, type=socket.SOCK_STREAM) 
returns (, , *, *, ('fe80::1', 80, 0, 1)). 
This feels like a dangerous assumption to make, since it's tied to the 
operating system's behavior. Maybe AIX's getaddrinfo() in fact does not resolve 
scoped addresses correctly; maybe it only resolves scope ids correctly for real 
addresses that actually exist on the network; Maybe AIX assigns scope ids 
differently and do not use small integers; etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36829] Add sys.unraisablehook() to custom how "unraisable exceptions" are logged

[STINNER Victor]

STINNER Victor  added the comment:


New changeset a58db9628d0c96cc5b863137fed4e432238f8027 by Victor Stinner in 
branch '3.7':
bpo-36829: Enhance PyErr_WriteUnraisable() (GH-13487)
https://github.com/python/cpython/commit/a58db9628d0c96cc5b863137fed4e432238f8027


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37003] ast unparse does not support f-string new debug format.

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Notice that test_tools will fail if  f'{x=}' becomes f'x={x!r}' when unparsed 
as it compares the text of the file and the text of the roundtrip of the ast of 
the file

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36817] Add = to f-strings for easier debugging.

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Anecdotally, this happened as well when in the implementation of PEP572

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36817] Add = to f-strings for easier debugging.

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The reason the main CI did not catch this is that test_tools is only executed 
on a (random) subset of all the files if I remember correctly because when 
executed on all files it massively increases the time of the CI.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37010] Review performance of inspect.getfullargspec

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
nosy: +pablogsal, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36929] Other Python _io implementations may not expose _io in their type names

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[STINNER Victor]

STINNER Victor  added the comment:

Michael Felt: it's annoying when you ignore Antoine's comment and my comment.
* https://github.com/python/cpython/pull/13463#issuecomment-494797084
* https://bugs.python.org/issue36084#msg343159

The AIX case is very special and required a separated issue. Please open a 
separated if you want to discuss/implement get_native_id() on AIX.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses

[twisteroid ambassador]

twisteroid ambassador  added the comment:

AFAIK the reason why scope id is required for IPv6 is that every IPv6
interfaces has its own link-local address, and all these addresses are in
the same subnet, so without an additional scope id there’s no way to tell
from which interface an address can be reached. IPv4 does not have this
problem because IPv4 interfaces usually don’t use link-local addresses.

Michael Felt 于2019年5月22日 周三18:08写道：

>
> Michael Felt  added the comment:
>
> On 22/05/2019 10:43, Michael Felt wrote:
> > 'fe80::1%1' <> 'fe80::1' - ... I am not 'experienced' with IPv6 and
> scope.
>
> >From what I have just read (again) - scope seems to be a way to indicate
> the interface used (e.g., eth0, or enp0s25) as a "number".
>
> Further, getsockname() (and getpeername()) seem to be more for after a
> fork(), or perhaps after a pthread_create(). What remains unclear is why
> would I ever care what the scopeid is.  Is it because it is "shiney",
> does it add security (if so, how)?
>
> And, as this has been added - what breaks in Python when "scopeid" is
> not available?
>
> I am thinking, if adding a scopeid is a way to assign an IPv6 address to
> an interface - what is to prevent abuse? Why would I even want the same
> (link-local IP address on eth0 and eth1 at the same time? Assuming that
> it what it is making possible - the same IPv6/64 address on multiple
> interfaces and use scope ID to be more selective/aware. It this an
> alternative way to multiplex interfaces - now in the IP layer rather
> than in the LAN layer?
>
> If I understand why this is needed I may be able to come up with a way
> to "get it working" for the Python model of interfaces - although,
> probably not "fast".
>
> Regards,
>
> Michael
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[Michael Felt]

Michael Felt  added the comment:

On 22/05/2019 15:15, Jake Tesler wrote:
> Jake Tesler  added the comment:
>
> I will look into whether adding thread_self() for AIX would be simple enough 
> for this PR.
>
> --
>
> ___
> Python tracker 
> 
> ___
>
Blush. Maybe I should have read further (to chapter T)

Here is a bare bones example - showing the pthread_self() is not the
right value, but thread_self is.

michael@x071:[/data/prj/aixtools/tests/posix]cat thread*.c
#include 
#include 
#include 

main()
{
    pid_t pid;
    tid_t tid;
    pthread_t ptid;

    pid = getpid();
    tid = thread_self();
    ptid = pthread_self();

    fprintf(stderr,"thread_self: pid:%d tid:%d ptid:%d\n", pid, tid,
ptid);
    sleep(300); /* give time to run ps -mo THREAD */
}

michael@x071:[/data/prj/aixtools/tests/posix]./thread_self
thread_self: pid:*4129010 *tid:*23724099 *ptid:1

michael@x071:[/data/prj/aixtools/tests/posix]ps -mo THREAD
    USER PID    PPID   TID S  CP PRI SC    WCHAN    F TT
BND COMMAND
 michael 3408006 7012502 - A   3  61  1    -   21 
pts/0  -1 ps -mo THREAD
   -   -   -  22282455 R   3  61  1    -   40 
-  -1 -
 michael *4129010 *7012502 - A   0  60  1 f1000a03e16533b0 
8200011  pts/0  -1 ./thread_self
   -   -   -  *23724099 *S   0  60  1 f1000a03e16533b0  
410400  -  -1 -

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36084] Threading: add builtin TID attribute to Thread objects

[STINNER Victor]

STINNER Victor  added the comment:

glibc 2.30 scheduled in August 2019 will finally provide a gettid() function!
* https://sourceware.org/git/?p=glibc.git;a=blob_plain;f=NEWS;hb=HEAD
* http://man7.org/linux/man-pages/man2/gettid.2.html

Once it will be released, it would be interesting to use it rather than using a 
direct syscall.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com