[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Kyle Evans : -- nosy: +kevans nosy_count: 8.0 -> 9.0 pull_requests: +21628 pull_request: https://github.com/python/cpython/pull/22651 ___ Python tracker ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Andrés Delfino : -- nosy: -adelfino ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Stefan Behnel : -- nosy: -scoder ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Stefan Behnel : -- pull_requests: -21520 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Stefan Behnel : -- nosy: +scoder nosy_count: 9.0 -> 10.0 pull_requests: +21520 pull_request: https://github.com/python/cpython/pull/22474 ___ Python tracker ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Andrés Delfino : -- nosy: +adelfino nosy_count: 8.0 -> 9.0 pull_requests: +21316 pull_request: https://github.com/python/cpython/pull/22205 ___ Python tracker ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by mohamed koubaa : -- nosy: +koubaa nosy_count: 7.0 -> 8.0 pull_requests: +21219 pull_request: https://github.com/python/cpython/pull/21986 ___ Python tracker ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Jakub Kulik : -- nosy: -kulikjak ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Jakub Kulik : -- pull_requests: -21198 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Jakub Kulik : -- nosy: +kulikjak nosy_count: 7.0 -> 8.0 pull_requests: +21198 pull_request: https://github.com/python/cpython/pull/22040 ___ Python tracker ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Gregory P. Smith : -- pull_requests: -2 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Raymond Hettinger : -- nosy: +rhettinger nosy_count: 6.0 -> 7.0 pull_requests: +2 pull_request: https://github.com/python/cpython/pull/21994 ___ Python tracker

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: New changeset 3b4b28efbde63502709bede7c5f9403ec6f37428 by Gregory P. Smith in branch '3.7': [3.7] bpo-36533: Reinit logging.Handler locks on fork(). (GH-12704) (GH-13170) https://github.com/python/cpython/commit/3b4b28efbde63502709bede7c5f9403ec6f37428

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: The regression should be fixed. It'd be helpful if owners of applications that were running into this could test their applications with this specific change. -- resolution: -> fixed stage: patch review -> commit review status: open -> closed

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Gregory P. Smith : -- pull_requests: +13086 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Gregory P. Smith : -- versions: -Python 3.8 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: New changeset 64aa6d2000665efb1a2eccae176df9520bf5f5e6 by Gregory P. Smith in branch 'master': bpo-36533: Reinit logging.Handler locks on fork(). (GH-12704) https://github.com/python/cpython/commit/64aa6d2000665efb1a2eccae176df9520bf5f5e6 --

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Łukasz Langa added the comment: This deadlock is a release blocker for 3.8.0. FTR, I don't consider it blocking alphas and betas. -- ___ Python tracker ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: The problem i am fixing is merely making Python's logging library not the source of this deadlock because it was not the source in the past. I am solving a regression in CPython behavior between 3.7.0 and 3.7.1 that led to a logging.Handler lock related

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

cagney added the comment: BTW, non-visible change might be to use a global readers-writer lock where fork() is the writer. https://en.wikipedia.org/wiki/Readers%E2%80%93writer_lock -- ___ Python tracker

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

cagney added the comment: > nope. the contrived emit() pseudocode from msg339650 never defined b in the > first place. that code, if massaged into python syntax would raise an > UnboundLocalError no matter who executed it. a fork() from another thread > would not change that. There is

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by STINNER Victor : -- nosy: -vstinner ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: > """ Since the documentation seems to be silent the guarantee (or expectation) is implied - logging systems emit records atomically - if this isn't true across fork() then the exception should be documented. """ We can add a note to the docs. As a

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

cagney added the comment: I pointed out two issues with breaking the locks. Your response addressed the less important issue: - the guarantee that log records don't interleave is lost Since the documentation seems to be silent the guarantee (or expectation) is implied - logging systems

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: The logging library has never guaranteed that it wouldn't interleave or duplicate buffered io output when fork() in involved. We should not start trying to claim that it does. Too complicated and fragile. It can't. Users who want that should implement

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

cagney added the comment: I think the only pragmatic solution here is to add an optional parameter to logging.basicConfig() that specifies that the logger should use a single global lock; and then start documenting that thread locks and fork() don't work well together. And note that this

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: Here's a PR that goes the re-initialization route rather than attempting to acquire/release anything other than the single module _lock. -- nosy: +lukasz.langa versions: +Python 3.8 ___ Python tracker

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Gregory P. Smith : -- keywords: +patch pull_requests: +12628 stage: -> patch review ___ Python tracker ___ ___

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Hugh Redelmeier : -- nosy: +hugh ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Change by Ned Deily : -- Removed message: https://bugs.python.org/msg339506 ___ Python tracker ___ ___ Python-bugs-list mailing

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Ebizz Infotech added the comment: The best-recommended method is hiring php web development services. https://www.ebizzinfotech.com/php-web-development/ -- nosy: +ebizzinfotech ___ Python tracker

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

cagney added the comment: On Fri, 5 Apr 2019 at 05:02, Gregory P. Smith wrote: > > > Gregory P. Smith added the comment: > > A stdlib alternative to this whole mess would be to avoid acquiring the > logging locks before fork() as we currently do and just blindly re-initialize > all of them

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

cagney added the comment: On Fri, 5 Apr 2019 at 04:15, Gregory P. Smith wrote: > > > New submission from Gregory P. Smith : > > I'm spawning a dicussion buried in the way too long thread of > https://bugs.python.org/issue6721 over here into its own specific issue to > treat as a 3.7 release

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: Now to back up: Why was this application using fork() in a threaded application at all? That is a fundamental flaw. Should we be doing work to support things that so far merely _happen_ to work on such broken designs? Another alternative for the

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: A stdlib alternative to this whole mess would be to avoid acquiring the logging locks before fork() as we currently do and just blindly re-initialize all of them afterwards under the assumption that they "can't" be protecting anything in a newly forked

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: Within the logging module we could go beyond using a WeakSet and maintain an ordering. But we'd need to allow a way for Handler subclasses to indicate what order matters; that'd require a new API (not suitable for 3.7) An ordering itself may be

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

Gregory P. Smith added the comment: That custom DebugHandler's emit() implementation that calls into one or more sub-handlers suggests that libreswan _might_ be able to fix it in the custom DebugHandler by implementing custom acquire() and release() methods... BUT that is a fundamentally

[issue36533] logging regression with threading + fork are mixed in 3.7.1rc2 (deadlock potential)

New submission from Gregory P. Smith : I'm spawning a dicussion buried in the way too long thread of https://bugs.python.org/issue6721 over here into its own specific issue to treat as a 3.7 release blocker for a rollback or repair decision before 3.7.4.