[issue16500] Allow registering at-fork handlers

2017-09-12 Thread STINNER Victor

STINNER Victor added the comment:


New changeset bcf042ff98b6261b7780c1e40fa1681ef30502f9 by Victor Stinner (Miss 
Islington (bot)) in branch '3.6':
[3.6] bpo-31234: Enhance test_thread.test_forkinthread() (GH-3516) (#3519)
https://github.com/python/cpython/commit/bcf042ff98b6261b7780c1e40fa1681ef30502f9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-09-12 Thread Roundup Robot

Changes by Roundup Robot :


--
pull_requests: +3516

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-09-12 Thread STINNER Victor

STINNER Victor added the comment:


New changeset a15d155aadfad232158f530278505cdc6f326f93 by Victor Stinner in 
branch 'master':
bpo-31234: Enhance test_thread.test_forkinthread() (#3516)
https://github.com/python/cpython/commit/a15d155aadfad232158f530278505cdc6f326f93


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-09-12 Thread STINNER Victor

Changes by STINNER Victor :


--
pull_requests: +3511

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-29 Thread Gregory P. Smith

Gregory P. Smith added the comment:

This may also allow us to do something reasonable for 
http://bugs.python.org/issue6721 as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-29 Thread STINNER Victor

STINNER Victor added the comment:

> New changeset 163468a766e16604bdea04a1ab808c0d3e729e5d by Gregory P. Smith in 
> branch 'master':
> bpo-16500: Don't use string constants for os.register_at_fork() behavior 
> (#1834)
> https://github.com/python/cpython/commit/163468a766e16604bdea04a1ab808c0d3e729e5d

Since there was a disagreement, FYI I also prefer this API. Just the
API, I didn't look at the implementation in detail.

Anyway, it's nice to see this very old issue finally fixed! It was
opened 5 years ago!

I also prefer to have a builtin registered callback in the random
module instead of having a custom callback in the multiprocessing
module to re-seed the RNG after fork!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-29 Thread Gregory P. Smith

Changes by Gregory P. Smith :


--
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-29 Thread Gregory P. Smith

Gregory P. Smith added the comment:


New changeset 163468a766e16604bdea04a1ab808c0d3e729e5d by Gregory P. Smith in 
branch 'master':
bpo-16500: Don't use string constants for os.register_at_fork() behavior (#1834)
https://github.com/python/cpython/commit/163468a766e16604bdea04a1ab808c0d3e729e5d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-29 Thread Antoine Pitrou

Antoine Pitrou added the comment:

Le 29/05/2017 à 18:35, Serhiy Storchaka a écrit :
> 
> Is it worth to guarantee the atomicity of os.register_at_fork().

I don't think so, honestly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-29 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

The one of potential advantages of the API proposed by Gregory is that 
os.register_at_fork() can be made atomic. Either register all callbacks, or do 
nothing in the case of error. But current proposed implementation is not 
atomic. If resizing of some list is failed due to MemoryError (or may be 
KeyboardInterrupt), some callbacks can already be registered.

Is it worth to guarantee the atomicity of os.register_at_fork().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Gregory P. Smith

Gregory P. Smith added the comment:

I'll just reuse this issue, I started that work just to try and clean up the 
new API and docs added in this one to be more modern.  Thanks for the code 
review, good suggestions!

--
assignee:  -> gregory.p.smith
stage: resolved -> commit review
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

In PR 1834 Gregory proposes an alternate API:

os.register_at_fork(*, before=None, after_in_parent=None, 
after_in_child=None)

Maybe open a new issue for this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Changes by Antoine Pitrou :


--
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Antoine Pitrou added the comment:


New changeset 4a8bcdf79cdb3684743fe1268de62ee88bada439 by Antoine Pitrou in 
branch 'master':
bpo-16500: Use register_at_fork() in the threading module (#1843)
https://github.com/python/cpython/commit/4a8bcdf79cdb3684743fe1268de62ee88bada439


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Antoine Pitrou added the comment:

> Can multiprocessing.util.register_after_fork() be rewritten with using the 
> new API?

It wouldn't benefit much from it, and there might be timing issue given the 
comments in BaseProcess._bootstrap():

old_process = _current_process
_current_process = self
try:
util._finalizer_registry.clear()
util._run_after_forkers()
finally:
# delay finalization of the old process object until after
# _run_after_forkers() is executed
del old_process

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Can multiprocessing.util.register_after_fork() be rewritten with using the new 
API?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Changes by Antoine Pitrou :


--
pull_requests: +1929

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Antoine Pitrou added the comment:

Le 28/05/2017 à 11:22, Serhiy Storchaka a écrit :
> 
> Serhiy Storchaka added the comment:
> 
> Can threading._after_fork() be rewritten with using the new API?

It should be possible indeed. Let me see.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Antoine Pitrou added the comment:


New changeset f7ecfac0c15f0c43ef5e6c0081eb3a059af9f074 by Antoine Pitrou in 
branch 'master':
Doc nits for bpo-16500 (#1841)
https://github.com/python/cpython/commit/f7ecfac0c15f0c43ef5e6c0081eb3a059af9f074


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Can threading._after_fork() be rewritten with using the new API?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Antoine Pitrou

Changes by Antoine Pitrou :


--
pull_requests: +1928

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-28 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

Could you please update the documentation Antoine?

* An entry in the C API section of Misc/NEWS.
* The "What's New" document needs changes in multiple places: the os module 
improvement, C API additions and deprecation, and may be in porting guide.
* Update references to PyOS_AfterFork() in the documentation and comments.

--
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-27 Thread Gregory P. Smith

Changes by Gregory P. Smith :


--
pull_requests: +1917

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-27 Thread Antoine Pitrou

Changes by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-27 Thread Antoine Pitrou

Antoine Pitrou added the comment:


New changeset 346cbd351ee0dd3ab9cb9f0e4cb625556707877e by Antoine Pitrou in 
branch 'master':
bpo-16500: Allow registering at-fork handlers (#1715)
https://github.com/python/cpython/commit/346cbd351ee0dd3ab9cb9f0e4cb625556707877e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Antoine Pitrou

Antoine Pitrou added the comment:

> If the corresponding warnings are ignored, nothing happen, if they are 
> errors, the child is exited,

Right now PyOS_AfterFork() doesn't return an error code.  It is not obvious how 
the caller would react: simply print the error? raise a fatal error? Something 
else?

The only third-party use of PyOS_AfterFork() I found is in uwsgi (and I'm not 
sure they're using it correctly, since I don't know if the parent process is 
using Python at all...).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

PyOS_AfterFork() is used by one code, but the "before" and "parent" handlers 
are registered by the other code (likely the code of other library). The author 
of the program that uses both libraries (the one that uses PyOS_AfterFork() and 
the one that registers handlers) can notice the warning and report the bug in 
the first library. I think silently skipping registered handlers would be worse.

Let allow the user to control the behavior by setting the warnings filter. If 
the corresponding warnings are ignored, nothing happen, if they are errors, the 
child is exited, otherwise the warning message is output on stderr as for other 
warnings.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Antoine Pitrou

Antoine Pitrou added the comment:

I don't think exiting would be a good idea at all.  I'm not sure about emitting 
a warning: the problem is that the people seeing the warning (plain users) 
can't do anything about it; worse, they probably will not even understand what 
it is about, and will get confused.

The package maintainer should see a deprecation warning when compiling code 
using PyOS_AfterFork() (thanks to the Py_DEPRECATED attribute).

Of course, I'll also update the docs to add a deprecation once the design is 
settled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

I agree that with lambdas and functools.partial the support of arguments is not 
needed. Unless someone has good reasons for supporting explicit passing of 
arguments I'm fine with your design.

If the user code manually calls the fork() and (now deprecated) 
PyOS_AfterFork(), it runs "child" functions, but not "before" and "parent" 
functions. Is it worth to emit a runtime warning if the "before" or "parent" 
lists are not empty? Or even immediately exit a child with error?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Antoine Pitrou

Antoine Pitrou added the comment:

API-wise, I went for the minimal route. This avoids any discussion of adding a 
separate module for a tiny functionality that is only going to be used by a 
couple of libraries (and probably no application code).

Comparisons with atexit are not really relevant, IMO, since the use cases are 
very different.

As for passing explicit arguments to the callable, people can use a lambda or 
functools.partial.  I don't want to complicate the C implementation with 
matters that are not really important.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Serhiy Storchaka

Serhiy Storchaka added the comment:

`atexit.register()` has different signature and allows to pass arbitrary 
positional and keyword arguments to the registered function. This is 
incompatible with the "when" argument. If we want to support registering 
arguments with the function, we need either three registering functions or make 
the "when" parameter the first and positional-only.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue16500] Allow registering at-fork handlers

2017-05-22 Thread Antoine Pitrou

Changes by Antoine Pitrou :


--
title: Add an 'atfork' module -> Allow registering at-fork handlers

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com