[issue40302] Add _Py_bswap32() function to pyport.h

2020-04-16 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Isn't pyport.h a public header?  IMHO you should put in a private header and 
make it an inline function there.

Here is for example what we do in Arrow:
https://github.com/apache/arrow/blob/master/cpp/src/arrow/util/bit_util.h#L48

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue40302>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2020-04-15 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Steve is right.

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2020-04-15 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I'll note that this "extremely common" model can break as soon as you have 
hidden worker threads somewhere (this can happen in a third-party library).

For example, the libhdfs library is not fork-safe.

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-15 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +18887
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/19539

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40286] Add getrandbytes() method to random.Random

2020-04-15 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +mark.dickinson

___
Python tracker 
<https://bugs.python.org/issue40286>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I agree I don't *need* it per se.  However, I suspect that for non-exports it 
would be easier than
`getrandbits(nbytes * 8).to_bytes(nbytes, 'endian')`.

As for `secrets.token_bytes()`, it's not really adequate for regular 
pseudo-random data generation when you want to use a fixed seed.  And I'm not 
sure what its performance characteristics are when you pass a large size.

--

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-14 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
components: +Library (Lib)

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-14 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
stage:  -> needs patch

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

About a hypothetical getrandbytes(), probably 90% of my uses of getrandbits() 
have been to generate random bytestrings.

--

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I think there's a misunderstanding: "wait_at_exit" will make the *executor* 
forget about the threads, but Python itself still knows about them, and it 
waits for them to end at interpreter shutdown.

These threads were daemon threads in 3.8, so your patch indeed works there, but 
we've made them non-daemon in 3.9, for two reasons:
1) daemon threads are fragile and can crash the interpreter at shutdown
2) they are not supported on subinterpreters

--

___
Python tracker 
<https://bugs.python.org/issue36780>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I don't think there's much ThreadPoolExecutor can do.  If you drop the 
references to the threads, they still exist and they still be waited upon at 
interpreter exit.

The solution is for you to avoid having hanging threads.  In the particular 
case of TCP connections, I'd recommend using a dedicated framework such as 
asyncio (or Twisted, Tornado, etc.) instead of home-baked networking code.

Also, note that Python sockets have a feature called *timeouts*.

--

___
Python tracker 
<https://bugs.python.org/issue36780>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36780] Interpreter exit blocks waiting for futures of shut-down ThreadPoolExecutors

2020-04-14 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue36780>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I think you know the answer to your question ;-)

--

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40282] random.getrandbits(0) should succeed

2020-04-14 Thread Antoine Pitrou


New submission from Antoine Pitrou :

When creating variable-sized random binary strings with random.getrandbits(), 
you currently have to special case when the number of bytes is 0, because 
otherwise getrandbits() raises:

  ValueError: number of bits must be greater than zero

It seems like it wouldn't hurt to simply return 0 in that case.

The actual snippet looks something like:

  random.getrandombits(nbytes * 8).to_bytes(nbytes, 'little')

--
messages: 366392
nosy: mark.dickinson, pitrou, rhettinger, steven.daprano
priority: normal
severity: normal
status: open
title: random.getrandbits(0) should succeed
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40282>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

As a separate discussion, I would be interested to know whether the original 
use case (Eddie's) could be satisfied differently.  It probably doesn't belong 
to this issue, though.

(Eddie, if you want to discuss this, feel free to e-mail me privately. I think 
Davin Potts - co-maintainer of multiprocessing who recently added some 
facilities around shared memory - might be interested as well.)

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40255] Fixing Copy on Writes from reference counting

2020-04-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

> I run the pyperformance test suite with PGO + LTO + full cpu isolation in the 
> speed.python.org machine and these were the results:

Be mindful that synthetic benchmarks are probably a gentle case for branch 
prediction.  Real-world performance on irregular workloads might be worse still 
(or perhaps better, who knows :-)).

--

___
Python tracker 
<https://bugs.python.org/issue40255>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40281] Add pathlib.PurePath.as_uri()

2020-04-14 Thread Antoine Pitrou


New submission from Antoine Pitrou :

"file" scheme URIs are normalized:
https://tools.ietf.org/html/rfc8089

Therefore it would be possible to provide a PurePath method that returns a 
"file" scheme URI corresponding to the given path.

--
components: Library (Lib)
messages: 366384
nosy: pitrou
priority: normal
severity: normal
status: open
title: Add pathlib.PurePath.as_uri()
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue40281>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue40281] Add pathlib.PurePath.as_uri()

2020-04-14 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +barneygale

___
Python tracker 
<https://bugs.python.org/issue40281>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39682] pathlib.Path objects can be used as context managers

2020-04-01 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
versions: +Python 3.9 -Python 3.8

___
Python tracker 
<https://bugs.python.org/issue39682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39682] pathlib.Path objects can be used as context managers

2020-04-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset 2e6d8b0ccdb6e0d9e98a9a7f9c9edfdf1311 by Barney Gale in branch 
'master':
bpo-39682: make `pathlib.Path` immutable by removing (undocumented) support for 
"closing" a path by using it as a context manager (GH-18846)
https://github.com/python/cpython/commit/2e6d8b0ccdb6e0d9e98a9a7f9c9edfdf1311


--

___
Python tracker 
<https://bugs.python.org/issue39682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39682] pathlib.Path objects can be used as context managers

2020-04-01 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue39682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39812] Avoid daemon threads in concurrent.futures

2020-03-27 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39812] Avoid daemon threads in concurrent.futures

2020-03-27 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset b61b818d916942aad1f8f3e33181801c4a1ed14b by Kyle Stanley in 
branch 'master':
bpo-39812: Remove daemon threads in concurrent.futures (GH-19149)
https://github.com/python/cpython/commit/b61b818d916942aad1f8f3e33181801c4a1ed14b


--

___
Python tracker 
<https://bugs.python.org/issue39812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39850] multiprocessing.connection.Listener fails to close with null byte in AF_UNIX socket name.

2020-03-08 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue39850>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39850] multiprocessing.connection.Listener fails to close with null byte in AF_UNIX socket name.

2020-03-08 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
stage:  -> needs patch
type: crash -> behavior
versions: +Python 3.7, Python 3.8, Python 3.9 -Python 3.6

___
Python tracker 
<https://bugs.python.org/issue39850>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39682] pathlib.Path objects can be used as context managers

2020-03-03 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Note you could simply remove the "closed" flag and the context manager live. 
That way, you don't have to deprecate anything.

--

___
Python tracker 
<https://bugs.python.org/issue39682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39682] pathlib.Path objects can be used as context managers

2020-03-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

As with the Accessor abstraction, the original idea was to support Path objects 
backed by a directory file descriptor (for use with openat() and friends).  
That idea was abandoned but it looks like the context manager stayed.  It's 
certainly not useful currently.

--

___
Python tracker 
<https://bugs.python.org/issue39682>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39778] collections.OrderedDict and weakref.ref raises "refcount is too small" assertion

2020-03-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Yes, I don't think other weakref-supporting objects traverse the weakreflist in 
their tp_traverse.

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue39778>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39825] EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig (Windows)

2020-03-02 Thread Antoine Pitrou


New submission from Antoine Pitrou :

On Windows, Python 3.7.6 and 3.8.1:
```
>>> import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.pyd'
>>> from distutils import sysconfig
>>> sysconfig.get_config_var('EXT_SUFFIX')
'.cp38-win_amd64.pyd'
```

The sysconfig answer is probably wrong (the ABI-qualified extension 
'.cp38-win_amd64.pyd' should be preferred).

--
components: Distutils, Library (Lib)
messages: 363171
nosy: dstufft, eric.araujo, paul.moore, pitrou, steve.dower, tarek, tim.golden, 
zach.ware
priority: normal
severity: normal
stage: needs patch
status: open
title: EXT_SUFFIX inconsistent between sysconfig and distutils.sysconfig 
(Windows)
type: behavior
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39825>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37266] Daemon threads must be forbidden in subinterpreters

2020-03-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Ok, I opened issue39812

--

___
Python tracker 
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15012] test issue

2020-03-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Unfortunately, I didn't receive the notification :-(

--

___
Python tracker 
<https://bugs.python.org/issue15012>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39678] RFC improve readability of _queue_management_worker for ProcessPoolExecutor

2020-03-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset 0e89076247580ba0e570c4816f0e5628a7e36e83 by Thomas Moreau in 
branch 'master':
bpo-39678: refactor queue manager thread (GH-18551)
https://github.com/python/cpython/commit/0e89076247580ba0e570c4816f0e5628a7e36e83


--

___
Python tracker 
<https://bugs.python.org/issue39678>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39678] RFC improve readability of _queue_management_worker for ProcessPoolExecutor

2020-03-01 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> enhancement

___
Python tracker 
<https://bugs.python.org/issue39678>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15012] test issue

2020-03-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Could someone please post a dummy reply to this issue? I'm hoping to fix e-mail 
notifications for me.

--

___
Python tracker 
<https://bugs.python.org/issue15012>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue15012] test issue

2020-03-01 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue15012>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39812] Avoid daemon threads in concurrent.futures

2020-03-01 Thread Antoine Pitrou


New submission from Antoine Pitrou :

Since issue37266 (which forbid daemon threads in subinterpreters), we probably 
want to forego daemon threads in concurrent.futures.  This means we also need a 
way to run an atexit-like hook before non-daemon threads are joined on 
(sub)interpreter shutdown.

See discussion below:
https://bugs.python.org/issue37266#msg362890

--
components: Library (Lib)
messages: 363059
nosy: aeros, pitrou, tomMoral
priority: normal
severity: normal
stage: needs patch
status: open
title: Avoid daemon threads in concurrent.futures
type: behavior
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39812>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39763] distutils.spawn should use subprocess (hang in parallel builds on QNX)

2020-03-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

@Elad:

> I'm not convinced that a multi-threaded fork()+exec() from C would be any 
> better, unless the Python code goes to great lengths to avoid any 
> non-async-signal-safe operations between the fork() and the exec().

That's exactly what the C code in the C subprocess module for POSIX does, 
though:

https://github.com/python/cpython/blob/master/Modules/_posixsubprocess.c#L828-L854

That's why I hope that using subprocess instead of a fork()+exec() sequence 
naively coded in pure Python would solve the issue.

@Guido:

> I can sort of see why you consider the small example a red herring, since it 
> mixes threads and fork

Perhaps "red herring" was the wrong expression.  What I mean is that it's no 
surprise that fork()+exec() sequence written in pure Python would be unsafe in 
multi-thread settings.  The solution, though, is not to avoid threads or try to 
workaround the issue in ThreadPoolExecutor, but rather to avoid doing 
fork()+exec() in pure Python.

--

___
Python tracker 
<https://bugs.python.org/issue39763>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39776] Crash in decimal module in heavy-multithreaded scenario

2020-03-01 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
priority: normal -> critical
versions: +Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39776>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37266] Daemon threads must be forbidden in subinterpreters

2020-02-29 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

> To me, this seems best implemented as a new public function for the threading 
> module, perhaps something like `threading.register_atexit()`. Would this be 
> reasonable?

To me, yes.

--

___
Python tracker 
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37266] Daemon threads must be forbidden in subinterpreters

2020-02-28 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Also cc'ing Kyle for the concurrent.futures issue.

--
nosy: +aeros

___
Python tracker 
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37266] Daemon threads must be forbidden in subinterpreters

2020-02-28 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Perhaps the solution would be to have an additional kind of atexit hooks, which 
get executed before threads are joined.

--
nosy: +ncoghlan

___
Python tracker 
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37266] Daemon threads must be forbidden in subinterpreters

2020-02-28 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

There will be a problem with `concurrent.futures.ProcessPoolExecutor`, which 
currently launches its management thread as a daemon thread.  The daemon thread 
itself is not problematic, because ProcessPoolExecutor uses an atexit hook to 
shutdown itself and therefore join the management thread.

It seems, however, that it's not easy to make the thread non-daemon, because 
atexit hooks are executed *after* non-daemon threads are joined.  That would 
lead to a deadlock: the interpreter would wait for the non-daemon management 
thread to exit, but the ProcessPoolExecutor would wait for the atexit hook to 
be called before telling the management thread to exit.

cc'ing Thomas Moreau, who's worker a lot on this.

--
nosy: +pitrou, tomMoral

___
Python tracker 
<https://bugs.python.org/issue37266>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39763] distutils.spawn should use subprocess (hang in parallel builds on QNX)

2020-02-26 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +dstufft, eric.araujo, p-ganssle, vstinner

___
Python tracker 
<https://bugs.python.org/issue39763>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39763] distutils.spawn should use subprocess (hang in parallel builds on QNX)

2020-02-26 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
title: Hang after fork due to logging trying to reacquire the module lock in an 
atfork() handler -> distutils.spawn should use subprocess (hang in parallel 
builds on QNX)

___
Python tracker 
<https://bugs.python.org/issue39763>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39763] Hang after fork due to logging trying to reacquire the module lock in an atfork() handler

2020-02-26 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Note the original problem stems when you using "setup.py -j ".

The attached reproducer is a bit of a red herring, because indeed doing fork() 
+ exec() in pure Python is too fragile.  That is why the standard subprocess 
does it in C.  The problem here is that the `distutils.spawn` module still has 
its hand-baked fork() + exec() pure Python code from the 1990s.

So the solution to the OP's actual problem should be to recode 
`distutils.spawn` using the subprocess module.

Original mailing-list discussion at:
https://mail.python.org/archives/list/python-...@python.org/thread/PTJ3UAZNYBYBYQ4WLMSERZR2XIVWISLM/

--
type:  -> behavior
versions: +Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39763>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39681] pickle.load expects an object that implements readinto

2020-02-24 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

The regression is fixed in git master and in the 3.8 branch. However, I would 
still advice against the idiom of handling a mmap object as a regular file when 
using pickle.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39734] Deprecate readinto() fallback path in _pickle.c

2020-02-23 Thread Antoine Pitrou


New submission from Antoine Pitrou :

In issue39681 we reestablished the fallback to read() when a file-like object 
doesn't provide readinto() in _pickle.c.  However, doing so leads to lower 
performance and all file-like object should nowadays provide readinto() (simply 
by deriving from the right base class - e.g. io.BufferedIOBase).

I propose to issue a DeprecationWarning when the fallback behaviour is 
selected, so that one day we can finally remove it.

--
components: Library (Lib)
messages: 362547
nosy: ogrisel, pierreglaser, pitrou
priority: low
severity: normal
status: open
title: Deprecate readinto() fallback path in _pickle.c
type: enhancement
versions: Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39734>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39712] Doc for `-X dev` option should mention PYTHONDEVMODE

2020-02-21 Thread Antoine Pitrou


New submission from Antoine Pitrou :

In the doc for `-X` options (https://docs.python.org/3/using/cmdline.html#id5), 
when an option can be triggered through an equivalent environment variable, 
that variable is mentioned.

An exception to that is `-X dev`, which can also be triggered by the 
PYTHONDEVMODE variable, but that isn't mentioned in the CLI doc.

Other missing environment variables in the CLI doc are PYTHONFAULTHANDLER and 
PYTHONTRACEMALLOC.

--
assignee: docs@python
components: Documentation
keywords: easy, newcomer friendly
messages: 362414
nosy: docs@python, eric.araujo, ezio.melotti, mdk, pitrou, vstinner, willingc
priority: normal
severity: normal
stage: needs patch
status: open
title: Doc for `-X dev` option should mention PYTHONDEVMODE
type: enhancement
versions: Python 3.7, Python 3.8, Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39712>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39681] pickle.load expects an object that implements readinto

2020-02-21 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
assignee:  -> pitrou
type:  -> behavior

___
Python tracker 
<https://bugs.python.org/issue39681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39681] pickle.load expects an object that implements readinto

2020-02-21 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +17960
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18592

___
Python tracker 
<https://bugs.python.org/issue39681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39681] pickle.load expects an object that implements readinto

2020-02-21 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Well, in the mmap case, this is trivially fixed by writing:
```
with open("my_data.pkl", "r+b") as f_in:
mm = mmap.mmap(f_in.fileno(), 0)

print(pickle.loads(memoryview(mm)))
```

It will also be faster this way, since the pickle will be read directly from 
memory instead of going through read() calls.

This makes me think that perhaps this issue isn't as bad: it forces people to 
abandon bad idioms :-)

--

___
Python tracker 
<https://bugs.python.org/issue39681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

2020-02-21 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Sorry for the delay.  I've now merged the PR for 3.9.  Since this is a slight 
behaviour change, and the original issue was easy to workaround, I won't 
backport it.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue35727>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35727] sys.exit() in a multiprocessing.Process does not align with Python behavior

2020-02-21 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset c2ac4cf040ea950bf552d1e77bea613a1a5474fe by Christopher Hunt in 
branch 'master':
bpo-35727: Use exit code 0 on sys.exit() in multiprocessing.Process. (GH-11538)
https://github.com/python/cpython/commit/c2ac4cf040ea950bf552d1e77bea613a1a5474fe


--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue35727>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39681] pickle.load expects an object that implements readinto

2020-02-19 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Right, I think at some point during the PEP 574 implementation review, we 
decided that the input object had to be a proper file implementation
(for example, deriving from io.BufferedIOBase).

I think it should be easy for PyTorch to fix this internally (by deriving from 
the proper io base class), but we can also add a fix to pickle.

--
versions: +Python 3.9

___
Python tracker 
<https://bugs.python.org/issue39681>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39104] ProcessPoolExecutor hangs on shutdown nowait with pickling failure

2020-02-16 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Thank you Thomas. Ii'd rather not backport delicate changes, so closing now.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior
versions:  -Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue39104>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue1294959] Problems with /usr/lib64 builds.

2020-02-12 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy:  -pitrou

___
Python tracker 
<https://bugs.python.org/issue1294959>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39492] reference cycle affecting Pickler instances (Python3.8+)

2020-02-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset 17236873392533ce0c5d7bbf52cbd61bca171c59 by Antoine Pitrou in 
branch '3.8':
[3.8] bpo-39492: Fix a reference cycle between reducer_override and a Pickler 
instance (GH-18266) (#18316)
https://github.com/python/cpython/commit/17236873392533ce0c5d7bbf52cbd61bca171c59


--

___
Python tracker 
<https://bugs.python.org/issue39492>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39492] reference cycle affecting Pickler instances (Python3.8+)

2020-02-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Thank you for the PR!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39492>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39492] reference cycle affecting Pickler instances (Python3.8+)

2020-02-02 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
pull_requests: +17692
pull_request: https://github.com/python/cpython/pull/18316

___
Python tracker 
<https://bugs.python.org/issue39492>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39349] Add "cancel_futures" parameter to concurrent.futures.Executor.shutdown()

2020-02-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset 339fd46cb764277cbbdc3e78dcc5b45b156bb6ae by Kyle Stanley in 
branch 'master':
bpo-39349: Add *cancel_futures* to Executor.shutdown() (GH-18057)
https://github.com/python/cpython/commit/339fd46cb764277cbbdc3e78dcc5b45b156bb6ae


--

___
Python tracker 
<https://bugs.python.org/issue39349>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39349] Add "cancel_futures" parameter to concurrent.futures.Executor.shutdown()

2020-02-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Thank you very muck, Kyle!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39349>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39435] pickle: inconsistent arguments pickle.py vs _pickle.c vs docs

2020-01-25 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Reopening because it seems my reading of the doc was wrong (took a backslash 
for a regular slash :-o).

--
resolution: not a bug -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 
<https://bugs.python.org/issue39435>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39435] pickle: inconsistent arguments pickle.py vs _pickle.c vs docs

2020-01-24 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

As mentioned on the attached PR, the first argument is positional, so it 
doesn't matter that the name in the docs is not the same as the name in the 
code.

The name "bytes_object" makes it clear which type of object is accepted, which 
makes it a better fit IMHO than "data".  Therefore, I'm going to close this 
issue.

--
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39435>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39426] Pickler docstring misstates default and highest protocols

2020-01-24 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39426>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39426] Pickler docstring misstates default and highest protocols

2020-01-22 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Good catch.  Do you want to submit a PR?

--

___
Python tracker 
<https://bugs.python.org/issue39426>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39207] concurrent.futures.ProcessPoolExecutor does not properly reap jobs and spawns too many workers

2020-01-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

It would certainly be better to start the worker processes on demand. It  
probably also requires careful thought about how to detect that more workers 
are required.

--

___
Python tracker 
<https://bugs.python.org/issue39207>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39291] "pathlib.Path.link_to()" and "pathlib.Path.symlink_to()" have reversed usage

2020-01-10 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Closing as won't fix.  If you feel strongly about this, I would suggest to 
bring the discussion on python-dev: 
https://mail.python.org/mailman3/lists/python-dev.python.org/

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue39291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39291] "pathlib.Path.link_to()" and "pathlib.Path.symlink_to()" have reversed usage

2020-01-10 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Ahah. Nice catch!

Well, it's a pity this got overlooked when we added Path.link_to().  But I'm 
afraid it's late to change it now, since this has been released, and changing 
the argument order would break existing code in potentially dangerous ways.

Note the original issue where this was added was issue26978.

--
nosy: +nanjekyejoannah

___
Python tracker 
<https://bugs.python.org/issue39291>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39193] Out-of-bound write in ceval.c:_PyEval_EvalFrameDefault

2020-01-03 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I don't think so. I'll let other confirm.

--
nosy: +christian.heimes, vstinner

___
Python tracker 
<https://bugs.python.org/issue39193>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39193] Out-of-bound write in ceval.c:_PyEval_EvalFrameDefault

2020-01-03 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +benjamin.peterson

___
Python tracker 
<https://bugs.python.org/issue39193>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue39193] Out-of-bound write in ceval.c:_PyEval_EvalFrameDefault

2020-01-03 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

It seems what you're saying is that one can crash Python by crafting 
deliberately incorrect bytecode.  If so, then I'm afraid it's something we 
won't consider fixing.

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue39193>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13601] sys.stderr should be line-buffered when stderr is not a TTY

2020-01-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Jendrik, thank you for fixing this!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue13601>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue13601] sys.stderr should be line-buffered when stderr is not a TTY

2020-01-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:


New changeset 5b9077134cd0535f21905d5f5195847526cac99c by Antoine Pitrou 
(Jendrik Seipp) in branch 'master':
bpo-13601: always use line-buffering for sys.stderr (GH-17646)
https://github.com/python/cpython/commit/5b9077134cd0535f21905d5f5195847526cac99c


--

___
Python tracker 
<https://bugs.python.org/issue13601>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38879] Reordered error checking in PyArena_New().

2020-01-01 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I'm rejecting this, as explained on the Github PR.

--
nosy: +pitrou
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38879>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36051] Drop the GIL during large bytes.join operations?

2019-12-21 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +inada.naoki

___
Python tracker 
<https://bugs.python.org/issue36051>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36051] Drop the GIL during large bytes.join operations?

2019-12-21 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

If there is a large chunk (e.g. several MBs), dropping the GIL during the 
memcpy of that chunk may be beneficial.  This kind of optimization may be 
applicable in other similar cases (such as extending a bytearray or a BytesIO 
object).

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue36051>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue36476] Runtime finalization assumes all other threads have exited.

2019-12-21 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

> 1. Python daemon threads

I think the answer is to document a bit more clearly that they can pose all 
kinds of problems.  Perhaps we could even display a visible warning when people 
create daemon threads.

> 2. Python threads created in atexit handlers

We could run the "join non-daemon threads" routine a *second time* after atexit 
handlers have been called.  It probably can't hurt (unless people do silly 
things?).

> 3. non-Python threads accessing the C-API

This one I don't know how to handle. By construction, a non-Python thread can 
do anything it wants, and we cannot add guards against this at the beginning of 
each C API function. I think that when someone calls the C API, we're clearly 
in the realm of "consenting adults".

--

___
Python tracker 
<https://bugs.python.org/issue36476>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

2019-11-20 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

(previous message deleted, I hadn't noticed the "reuse_port" parameter)

My preference for create_datagram_endpoint() would be:

- make the "reuse_address" parameter a no-op, and raise an error when 
"reuse_address=True" is passed

- do that in 3.8 as well

This way we 1) solve the security issue 2) ensure that anyone that passed 
"reuse_address=True" explicitly is notified that they were doing something 
dangerous unwillingly.

And, yes, someone who really wants SO_REUSEADDR can set it manually, for 
example by calling `transport.get_extra_info('socket')`.

--

___
Python tracker 
<https://bugs.python.org/issue37228>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

2019-11-20 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
Removed message: https://bugs.python.org/msg357067

___
Python tracker 
<https://bugs.python.org/issue37228>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37228] UDP sockets created by create_datagram_endpoint() allow by default multiple processes to bind the same port

2019-11-20 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Here is another suggestion:

- make the "reuse_address" parameter a no-op, and raise an error when 
"reuse_address=True" is passed

- add a new "allow_multiple_bind=False" parameter that enables SO_REUSEPORT

- do that in 3.8 as well

This way we 1) solve the security issue 2) ensure that anyone that passed 
"reuse_address=True" explicitly is notified that they were doing something 
dangerous unwillingly.

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue37228>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38707] Multiprocessing: bug with Native ID for threading.mainthread()

2019-11-19 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Thank you Jake for the report and PR!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.9

___
Python tracker 
<https://bugs.python.org/issue38707>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38744] python 3.8 hang in multiprocessing.Pool() locking on FreeBSD

2019-11-13 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy: +pablogsal

___
Python tracker 
<https://bugs.python.org/issue38744>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38744] python 3.8 hang in multiprocessing.Pool() locking on FreeBSD

2019-11-13 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

What happens if you instead write:

with Pool() as pool:
pool.map(sleep, [0.01] * 10)

--

___
Python tracker 
<https://bugs.python.org/issue38744>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38422] Clarify docstrings of pathlib suffix(es)

2019-11-02 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38422>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38422] Clarify docstrings of pathlib suffix(es)

2019-11-02 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
nosy:  -miss-islington
versions: +Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue38422>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38279] multiprocessing example enhancement

2019-11-02 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I rejected the accompanying PR and the changes it proposed, therefore it seems 
reasonable to close this issue as well.

--
nosy: +pitrou
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38279>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38167] O_DIRECT read fails with 4K mmap buffer

2019-10-04 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Also I'm curious: why are you using O_DIRECT, and furthermore, why are you 
using it to read into mmap'ed memory?

--

___
Python tracker 
<https://bugs.python.org/issue38167>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38167] O_DIRECT read fails with 4K mmap buffer

2019-10-04 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I agree with Josh. If you want to use O_DIRECT, use an unbuffered file object 
and be sure to issue reads of the right size.

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue38167>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38249] Optimize out Py_UNREACHABLE in the release mode

2019-10-04 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I agree with Serhiy.  If you hit a rare situation that you don't want to 
handle, use Py_FatalError(), not Py_UNREACHABLE().

Py_UNREACHABLE() is for situations which are *logically* impossible, not just 
uncommon.

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue38249>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38284] signal.sigwait* do not intercept certain signals

2019-10-04 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Closing as not a Python bug.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue38284>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38284] signal.sigwait* do not intercept certain signals

2019-10-04 Thread Antoine Pitrou

Antoine Pitrou  added the comment:

Python just exposes thin wrappers around the underlying libc calls, so you have 
to understand how those work.

On Linux, the sigwaitinfo() man page says:

NOTES
   In normal usage, the calling program blocks the signals  in
   set via a prior call to sigprocmask(2) (so that the default
   disposition for these signals does not occur if they become
   pending  between  successive calls to sigwaitinfo() or sig‐
   timedwait()) and does not establish handlers for these sig‐
   nals.

So you need to block the given signal with pthread_sigmask() before waiting on 
it.  For example:

>>> import signal
>>> signal.pthread_sigmask(signal.SIG_BLOCK, [signal.SIGHUP])
set()
>>> signal.sigwait([signal.SIGHUP])


--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue38284>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38227] Setting a signal handler gets multiprocessing.Pool stuck

2019-10-04 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Several things here:

- you can perform critical cleanup with the atexit module; using a signal 
handler for that is extremely low-level and error-prone

- you can also try to switch to the "forkserver" method of multiprocessing, 
perhaps that will fix your issue

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue38227>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

2019-10-03 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

> Would it make any sense to add an opt-in option to emit a warning when a new 
> type is created with Py_TPFLAGS_HAVE_GC but it doesn't implement tp_traverse?

Or even fatal error out?  There's no reason to have Py_TPFLAGS_HAVE_GC but not 
implement tp_traverse, AFAIR.

--

___
Python tracker 
<https://bugs.python.org/issue38006>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37635] Using constant for whence arg in seek()

2019-09-15 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

> What do you think of the followup PR to make use of the SEEK_* constants 
> listed in the documentation? I think it would be useful to at least mention 
> them in the tutorial, or even make use of them directly in the examples.

Yes, I think it would be good to make use of them.

--

___
Python tracker 
<https://bugs.python.org/issue37635>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37635] Using constant for whence arg in seek()

2019-09-14 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Thanks you Kyle!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue37635>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37979] Document an alternative to ISO 8601 parsing

2019-08-29 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Thanks for contributing this, Paul!

--
nosy: +pitrou
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue37979>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue32847] Add DirectoryNotEmptyError subclass of OSError

2019-08-29 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

According to POSIX, the only system calls that can fail with ENOTEMPTY are 
rmdir(), rename() and unlinkat().

--

___
Python tracker 
<https://bugs.python.org/issue32847>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37790] subprocess.Popen() is extremely slow

2019-08-17 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

I don't seen any significant difference here (Ubuntu 18.04):

$ time python2 sp.py 
10.3433089256

real0m10,362s
user0m6,565s
sys 0m4,372s

$ time python3 sp.py 
11.746907234191895

real0m11,781s
user0m7,356s
sys 0m5,239s

--

___
Python tracker 
<https://bugs.python.org/issue37790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37790] subprocess.Popen() is extremely slow

2019-08-16 Thread Antoine Pitrou


Antoine Pitrou  added the comment:

Can you post a simple script showcasing the performance degradation?

--
nosy: +pitrou

___
Python tracker 
<https://bugs.python.org/issue37790>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue37689] Add Path.is_relative_to()

2019-08-13 Thread Antoine Pitrou


Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 
<https://bugs.python.org/issue37689>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



<    1   2   3   4   5   6   7   8   9   10   >