[issue6721] Locks in the standard library should be sanitized on fork

2020-10-11 Thread Kyle Evans
Change by Kyle Evans : -- nosy: +kevans nosy_count: 28.0 -> 29.0 pull_requests: +21627 pull_request: https://github.com/python/cpython/pull/22651 ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Andrés Delfino
Change by Andrés Delfino : -- nosy: -adelfino ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Stefan Behnel
Change by Stefan Behnel : -- nosy: -scoder ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Stefan Behnel
Change by Stefan Behnel : -- pull_requests: -21519 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-10-03 Thread Stefan Behnel
Change by Stefan Behnel : -- nosy: +scoder nosy_count: 29.0 -> 30.0 pull_requests: +21519 pull_request: https://github.com/python/cpython/pull/22474 ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-09-15 Thread Andrés Delfino
Change by Andrés Delfino : -- nosy: +adelfino nosy_count: 28.0 -> 29.0 pull_requests: +21315 pull_request: https://github.com/python/cpython/pull/22205 ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-09-07 Thread mohamed koubaa
Change by mohamed koubaa : -- nosy: +koubaa nosy_count: 27.0 -> 28.0 pull_requests: +21218 pull_request: https://github.com/python/cpython/pull/21986 ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-09-07 Thread Jakub Kulik
Change by Jakub Kulik : -- pull_requests: -21197 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-09-07 Thread Jakub Kulik
Change by Jakub Kulik : -- nosy: -kulikjak ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-09-05 Thread Jakub Kulik
Change by Jakub Kulik : -- nosy: +kulikjak nosy_count: 27.0 -> 28.0 pull_requests: +21197 pull_request: https://github.com/python/cpython/pull/22040 ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-08-29 Thread Gregory P. Smith
Change by Gregory P. Smith : -- pull_requests: -21110 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-08-29 Thread Raymond Hettinger
Change by Raymond Hettinger : -- nosy: +rhettinger nosy_count: 26.0 -> 27.0 pull_requests: +21110 pull_request: https://github.com/python/cpython/pull/21994 ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2020-06-11 Thread Jesse Farnham
Change by Jesse Farnham : -- nosy: +jesse.farnham ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2020-05-14 Thread STINNER Victor
STINNER Victor added the comment: See also bpo-25920: PyOS_AfterFork should reset socketmodule's lock. -- ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-04-29 Thread Deomid Ryabkov
Deomid Ryabkov added the comment: https://bugs.python.org/issue40442 is a fresh instance of this, entirely self-inflicted. -- nosy: +rojer ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2020-04-28 Thread Antoine Pitrou
Antoine Pitrou added the comment: Related issue: https://bugs.python.org/issue40399 """ IO streams locking can be broken after fork() with threads """ -- ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2020-03-27 Thread STINNER Victor
STINNER Victor added the comment: I created bpo-40089: Add _at_fork_reinit() method to locks. -- ___ Python tracker ___ ___

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-05 Thread Hugh Redelmeier
Change by Hugh Redelmeier : -- nosy: +hugh ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-05 Thread Gregory P. Smith
Gregory P. Smith added the comment: Thanks for the debugging details! I've filed https://bugs.python.org/issue36533 to specifically track this potential regression in the 3.7 stable branch. lets carry on there where the discussion thread isn't too long for bug tracker sanity. --

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-04 Thread A. Jesse Jiryu Davis
Change by A. Jesse Jiryu Davis : -- nosy: -emptysquare ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-04 Thread cagney
cagney added the comment: Below is a backtrace from the deadlock. It happens because the logging code is trying to acquire two per-logger locks; and in an order different to the random order used by the fork() handler. The code in question has a custom class DebugHandler(logging.Handler).

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-04 Thread cagney
cagney added the comment: > acquiring locks before fork in the thread doing the forking and releasing > them afterwards is always the safe thing to do. It's also an easy way to cause a deadlock: - register_at_fork() et.al. will cause per-logger locks to be acquired before the global lock

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-03 Thread Gregory P. Smith
Gregory P. Smith added the comment: I'd start with faulthandler.register with all_threads=True and see if that gives you what you need. https://docs.python.org/3/library/faulthandler.html -- ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-03 Thread cagney
cagney added the comment: > Does your code use any C code that forks on its own without properly calling > the C Python PyOS_BeforeFork(), PyOS_AfterFork_Parent(), and > PyOS_AfterFork_Child() APIs? No. Is there a web page explaining how to pull a python backtrace from all the threads

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-02 Thread Gregory P. Smith
Gregory P. Smith added the comment: We need a small test case that can reproduce your problem. I believe https://github.com/python/cpython/commit/3b699932e5ac3e76031bbb6d700fbea07492641d to be correct. acquiring locks before fork in the thread doing the forking and releasing them

[issue6721] Locks in the standard library should be sanitized on fork

2019-04-02 Thread cagney
cagney added the comment: I suspect 3b699932e5ac3e7 is causing a hang in libreswan's kvmrunner.py on Fedora. Looking at the Fedora RPMs: python3-3.7.0-9.fc29.x86_64 didn't contain the fix and worked python3-3.7.1-4.fc29.x86_64 reverted the fix (for anaconda) and worked

[issue6721] Locks in the standard library should be sanitized on fork

2018-11-08 Thread STINNER Victor
STINNER Victor added the comment: > New changeset 3b699932e5ac3e76031bbb6d700fbea07492641d by Gregory P. Smith > (Miss Islington (bot)) in branch '3.7': > bpo-6721: Hold logging locks across fork() (GH-4071) (#9291) It seems like this change caused a regression in the Anaconda installer of

[issue6721] Locks in the standard library should be sanitized on fork

2018-10-07 Thread Gregory P. Smith
Gregory P. Smith added the comment: New changeset 3b699932e5ac3e76031bbb6d700fbea07492641d by Gregory P. Smith (Miss Islington (bot)) in branch '3.7': bpo-6721: Hold logging locks across fork() (GH-4071) (#9291)

[issue6721] Locks in the standard library should be sanitized on fork

2018-09-13 Thread miss-islington
Change by miss-islington : -- pull_requests: +8722 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue6721] Locks in the standard library should be sanitized on fork

2018-09-13 Thread Gregory P. Smith
Gregory P. Smith added the comment: New changeset 19003841e965bbf56fd06824d6093620c1b66f9e by Gregory P. Smith in branch 'master': bpo-6721: Hold logging locks across fork() (GH-4071) https://github.com/python/cpython/commit/19003841e965bbf56fd06824d6093620c1b66f9e --

[issue6721] Locks in the standard library should be sanitized on fork

2018-04-05 Thread Olivier Chédru
Olivier Chédru added the comment: FWIW, I encountered the same kind of issue when using the mkstemp() function: under the hood, it calls gettempdir() and this one is protected by a lock too. Current thread 0x7ff10231f700 (most recent call first): File

[issue6721] Locks in the standard library should be sanitized on fork

2017-10-21 Thread Antoine Pitrou
Antoine Pitrou added the comment: Actually, we already have a doubly-linked list of buffered IO objects (for another purpose), so we can reuse that and register a single set of global callbacks. -- ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2017-10-21 Thread Gregory P. Smith
Gregory P. Smith added the comment: logging is pretty easy to deal with so I created a PR. bufferedio.c is a little more work as we either need to use the posixmodule.c os.register_at_fork API or expose it as an internal C API to be able to call it to add acquires and

[issue6721] Locks in the standard library should be sanitized on fork

2017-10-21 Thread Gregory P. Smith
Change by Gregory P. Smith : -- pull_requests: +4042 ___ Python tracker ___ ___

[issue6721] Locks in the standard library should be sanitized on fork

2017-10-21 Thread Antoine Pitrou
Antoine Pitrou added the comment: Oh, I forgot that IO buffered objects also have a lock. So we would have to special-case those as well, unless we take the generic approach... A problem with the generic approach is that it would leave higher-level synchronization objects

[issue6721] Locks in the standard library should be sanitized on fork

2017-10-21 Thread Antoine Pitrou
Antoine Pitrou added the comment: I think we should somehow move forward on this, at least for logging locks which can be quite an annoyance. There are two possible approaches: - either a generic mechanism as posted by sbt in reinit_locks_2.diff - or a logging-specific fix

[issue6721] Locks in the standard library should be sanitized on fork

2017-05-31 Thread Daniel Birnstiel
Daniel Birnstiel added the comment: While having to deal with this bug for a while I have written a small library using `pthread_atfork`: https://github.com/Birne94/python-atfork-lock-release It allows registering atfork-hooks (similar to the ones available by now) and frees the stdout/stderr

[issue6721] Locks in the standard library should be sanitized on fork

2017-05-29 Thread Gregory P. Smith
Gregory P. Smith added the comment: http://bugs.python.org/issue16500 added the os.register_at_fork() API which may be usable for this. -- ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2017-05-29 Thread Gregory P. Smith
Changes by Gregory P. Smith : -- versions: +Python 3.7 -Python 3.5 ___ Python tracker ___ ___

[issue6721] Locks in the standard library should be sanitized on fork

2017-03-16 Thread Daniel Birnstiel
Daniel Birnstiel added the comment: Currently using Python 3.6.0 (default, Mar 4 2017, 12:32:34) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin > So, somehow the print() statement is blocking, which I have /no/ idea how to > go about debugging. I assume there's a lock

[issue6721] Locks in the standard library should be sanitized on fork

2017-02-17 Thread Camilla Montonen
Changes by Camilla Montonen : -- nosy: +Winterflower ___ Python tracker ___ ___

[issue6721] Locks in the standard library should be sanitized on fork

2016-09-07 Thread Davin Potts
Changes by Davin Potts : -- nosy: +davin ___ Python tracker ___ ___ Python-bugs-list

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-09 Thread STINNER Victor
STINNER Victor added the comment: I suggest to close the issue as WONT FIX. Python code base is huge and Python depends on a lot of external code. We cannot warranty anything. It might be possible to track all kinds of locks with an infinite time. But I'm not sure that it's worth it. It is

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread Connor Wolf
Connor Wolf added the comment: > Python 3.5.1+ (default, Mar 30 2016, 22:46:26) Whatever the stock 3.5 on ubuntu 16.04 x64 is. I've actually been running into a whole horde of really bizarre issues related to what I /think/ is locking in stdout. Basically, I have a context where I have

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread Gregory P. Smith
Gregory P. Smith added the comment: My intent is not to block anything. I'm Just explaining why I'm not motivated to spend much time on this issue myself. Others are welcome to. subprocess is not related to this issue, it has been fixed for use with threads (in 3.2 and higher) with an

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread lesha
lesha added the comment: On a moment's reflection, a lot of my prior comment is wrong. Sorry about that. - glog does not, that I know of, sanitize locks on fork. You just shouldn't log after fork but before exec. - Using `pthread_atfork` to clean up the `logging` lock might be enough to make

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread lesha
lesha added the comment: I work on a multi-million-line C++ codebase that uses fork() from multithreaded programs all over the place. We use `glog` ubiquitously. This bug here that spans 6 years and has affected dozens of people (conservatively) simply does not exist for us. That is because

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread Connor Wolf
Connor Wolf added the comment: Arrrgh, s/threading/multiprocessing/g in my last message. -- ___ Python tracker ___

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread Connor Wolf
Connor Wolf added the comment: > IMNSHO, working on "fixes" for this issue while ignoring the larger > application design flaw elephant in the room doesn't make a lot of sense. I understand the desire for a canonically "correct" fix, but it seems the issue with fixing it "correctly" has lead

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread Gregory P. Smith
Gregory P. Smith added the comment: For me the momentum on fixing these things has stalled because I no longer work on any code that runs into this. There is a fundamental problem: You cannot safely use threading and os.fork() in the same application per POSIX rules. So even if the standard

[issue6721] Locks in the standard library should be sanitized on fork

2016-07-08 Thread Connor Wolf
Connor Wolf added the comment: Is anything happening with these fixes? This is still an issue (I'm running into it now)? -- nosy: +Connor.Wolf ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2016-03-24 Thread A. Jesse Jiryu Davis
Changes by A. Jesse Jiryu Davis : -- nosy: +emptysquare ___ Python tracker ___ ___

[issue6721] Locks in the standard library should be sanitized on fork

2014-11-03 Thread Nir Soffer
Changes by Nir Soffer nir...@gmail.com: -- nosy: +nirs ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list mailing list

[issue6721] Locks in the standard library should be sanitized on fork

2014-11-02 Thread Maries Ionel Cristian
Changes by Maries Ionel Cristian ionel...@gmail.com: -- nosy: +ionel.mc ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___

[issue6721] Locks in the standard library should be sanitized on fork

2014-08-23 Thread Dan O'Reilly
Changes by Dan O'Reilly oreil...@gmail.com: -- nosy: +dan.oreilly ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list

[issue6721] Locks in the standard library should be sanitized on fork

2014-06-30 Thread Tshepang Lekhonkhobe
Changes by Tshepang Lekhonkhobe tshep...@gmail.com: -- title: Locks in python standard library should be sanitized on fork - Locks in the standard library should be sanitized on fork versions: +Python 3.5 -Python 3.3 ___ Python tracker

[issue6721] Locks in the standard library should be sanitized on fork

2014-06-30 Thread Tshepang Lekhonkhobe
Changes by Tshepang Lekhonkhobe tshep...@gmail.com: -- nosy: +tshepang ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list

[issue6721] Locks in python standard library should be sanitized on fork

2014-04-22 Thread Forest Bond
Changes by Forest Bond for...@alittletooquiet.net: -- nosy: +forest_atq ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-02 Thread Tomaž Šolc
Changes by Tomaž Šolc tomaz.s...@tablix.org: -- nosy: -avian ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list mailing

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-02 Thread Vinay Sajip
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Use file locks in logging, whenever possible. Logging doesn't just log to files, and moreover, also has locks to serialise access to internal data structures (nothing to do with files). Hence, using file locks in logging is not going to

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-02 Thread Richard Oudkerk
Richard Oudkerk shibt...@gmail.com added the comment: Lesha, the problems about magical __del__ methods you are worried about actually have nothing to do with threading and locks. Even in a single threaded program using fork, exactly the same issues of potential corruption would be present

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread Gregory P. Smith
Gregory P. Smith g...@krypto.org added the comment: We could make any later attempt to acquire or release a lock that was reinitialized while it was held raise an exception. Such exception raising behavior should be conditional at lock construction time; some code (such as logging) never

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread Richard Oudkerk
Richard Oudkerk shibt...@gmail.com added the comment: conn = MySQLConn() start_thread1(conn) start_thread2(conn): while True: if os.fork() == 0: # child raise Exception('doom') # triggers destructor There is no guarantee here that the lock will be held at the time of the

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread Vinay Sajip
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Re threading locks cannot be used to protect things outside of a single process: The Python standard library already violates this, in that the logging module uses such a lock to protect the file/socket/whatever, to which it is

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: I feel like I'm failing to get my thesis across. I'll write it out fully: == Thesis start == Basic fact: It is an error to use threading locks in _any_ way after a fork. I think we mostly agree on this. The programs we discussing are

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: A slightly more ambitious solution than crashing / deadlocking always is to have Python automatically spawn a fork server whenever you start using threads. Then, you would be able to have subprocess work cleanly, and not worry about any of

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: Actually, we might be able to automatically spawn a safe fork server _only_ when people start mixing threading and subprocess. I'm not totally sure if this would allow us to salvage multiprocessing as well... The tricky bit is that we'd need

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread Gregory P. Smith
Gregory P. Smith g...@krypto.org added the comment: subprocess has nothing to do with this bug. subprocess is safe as of Python 3.2 (and the subprocess32 backport for 2.x). Its preexec_fn argument is already documented as an unsafe legacy. If you want to replace subprocess, go ahead, write

[issue6721] Locks in python standard library should be sanitized on fork

2012-06-01 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: 1) I'm totally in favor of making the standard library safe. For that purpose, I think we should do a combination of: a) Use file locks in logging, whenever possible. b) Introduce LockUnsafelyReinitializedAtFork, using a generation counter, or

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread Richard Oudkerk
Richard Oudkerk shibt...@gmail.com added the comment: Attached is an updated version of Charles-François's reinit_locks.diff. Changes: * Handles RLock by assuming that if self-count != 0 when we acquire the lock, then the lock must have been reinitialized by PyThread_ReInitLocks(). *

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: I am really alarmed by the reinit_locks patches. I scanned the comment history, and looked at the patch. I may have missed something, but it looks to me like the basic behavior is this: After fork(), all locks are replaced by brand-new lock

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread Richard Oudkerk
Richard Oudkerk shibt...@gmail.com added the comment: a) fork() is called with the DB lock held by thread1. b) Some time passes before the child gets to exec(). c) In that time, the child's thread2 gets to doWork(). d) Simultaneously, the parent's doWork is still running and holding a lock.

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: I think forkall() on Solaris acts like that, but the normal fork() function does not. Only the thread which performs fork() will survive in the child process. Sorry, brain fail. A slightly more contrived failure case is this:

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread Gregory P. Smith
Gregory P. Smith g...@krypto.org added the comment: Anyone using a preexec function in subprocess has already declared that they like deadlocks so that isn't an issue. :) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: Deadlocks are dandy, but corruption is cruel. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread Gregory P. Smith
Gregory P. Smith g...@krypto.org added the comment: threading locks cannot be used to protect things outside of a single process. Any code using them to do that is broken. In your examples you are suggesting a class that wants to do one or more mysql actions within a destructor and worried

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-31 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: Re threading locks cannot be used to protect things outside of a single process: The Python standard library already violates this, in that the logging module uses such a lock to protect the file/socket/whatever, to which it is writing. If I

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-30 Thread Richard Oudkerk
Richard Oudkerk shibt...@gmail.com added the comment: Is there any particular reason not to merge Charles-François's reinit_locks.diff? Reinitialising all locks to unlocked after a fork seems the only sane option. I agree with this. I haven't looked at the patch very closely. I

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-23 Thread Richard Oudkerk
Richard Oudkerk shibt...@gmail.com added the comment: (1) Good catch. I suspect that this could be mitigated even if we cared about LinuxThreads. I haven't looked, but there's got to be a way to determine if we are a thread or a fork child. Using a generation count would probably work just

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-23 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: So what are you suggesting? That a lock of the default type should raise an error if you try to acquire it when it has been acquired in a previous process? I was suggesting a way to make 'logging' fork-safe. No more, no less. Does what my

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-22 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: This is a reply to: http://bugs.python.org/issue6721#msg151266 Charles-François raises two problems: 1) LinuxThreads have different PIDs per thread. He then points out that LinuxThreads have long been deprecated. 2) you cannot release locks on

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-17 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Should we go forward on this? -- assignee: gregory.p.smith - stage: test needed - patch review type: behavior - enhancement versions: -Python 2.7, Python 3.2 ___ Python tracker rep...@bugs.python.org

[issue6721] Locks in python standard library should be sanitized on fork

2012-05-17 Thread Gregory P. Smith
Gregory P. Smith g...@krypto.org added the comment: going forward with reinit_locks.diff makes sense. I've added comments to it in the code review link. It is Patch Set 3 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721

[issue6721] Locks in python standard library should be sanitized on fork

2012-02-13 Thread Vinay Sajip
Changes by Vinay Sajip vinay_sa...@yahoo.co.uk: -- nosy: +vinay.sajip ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-23 Thread sbt
sbt shibt...@gmail.com added the comment: Attached is a patch (without documentation) which creates an atfork module for Unix. Apart from the atfork() function modelled on pthread_atfork() there is also a get_fork_lock() function. This returns a recursive lock which is held whenever a child

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-23 Thread sbt
sbt shibt...@gmail.com added the comment: Is there any particular reason not to merge Charles-François's reinit_locks.diff? Reinitialising all locks to unlocked after a fork seems the only sane option. -- ___ Python tracker rep...@bugs.python.org

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-23 Thread Antoine Pitrou
Antoine Pitrou pit...@free.fr added the comment: Is there any particular reason not to merge Charles-François's reinit_locks.diff? Reinitialising all locks to unlocked after a fork seems the only sane option. I agree with this. I haven't looked at the patch very closely. I think perhaps

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-14 Thread Jesús Cea Avión
Changes by Jesús Cea Avión j...@jcea.es: -- nosy: +jcea ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue6721 ___ ___ Python-bugs-list mailing list

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-14 Thread Charles-François Natali
Charles-François Natali neolo...@free.fr added the comment: However, extending RLock to provide ForkClearedRLock (this would be used by logging, i.e.) is quite straighforward. The extended class would simply need to record the process ID, in which the lock was created, and the process ID,

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-14 Thread Gregory P. Smith
Gregory P. Smith g...@krypto.org added the comment: A new lock type will NOT solve this. It is ALWAYS okay to clear all thread/threading module locks after a fork. They are and always have been process-local by definition so they are also by definition 100% invalid to any child process.

[issue6721] Locks in python standard library should be sanitized on fork

2012-01-13 Thread lesha
lesha pybug.20.le...@xoxy.net added the comment: Just wanted to say that I spent something like 8 hours debugging a subprocess + threading + logging deadlock on a real production system. I suspected one of my locks at first, but I couldn't find any. The post-fork code was very simple, and I

[issue6721] Locks in python standard library should be sanitized on fork

2011-08-31 Thread Nir Aides
Nir Aides n...@winpdb.org added the comment: For the record, turns out there was a bit of misunderstanding. I used the term deprecate above to mean warn users (through documentation) that they should not use (a feature) and not in its Python-dev sense of remove (a feature) after a period of

[issue6721] Locks in python standard library should be sanitized on fork

2011-08-31 Thread Charles-François Natali
Charles-François Natali neolo...@free.fr added the comment: Anyway, since my view does not seem to resonate with core developers I I'll give it a rest for now. Well, the problem is that many views have been expressed in this thread, which doesn't help getting a clear picture of what's needed

[issue6721] Locks in python standard library should be sanitized on fork

2011-08-29 Thread sbt
sbt shibt...@gmail.com added the comment: multiprocessing.util already has register_after_fork() which it uses for cleaning up certain things when a new process (launched by multiprocessing) is starting. This is very similar to the proposed atfork mechanism. Multiprocessing assumes that it

[issue6721] Locks in python standard library should be sanitized on fork

2011-07-28 Thread Nir Aides
Nir Aides n...@winpdb.org added the comment: Hi Gregory, Gregory P. Smith g...@krypto.org added the comment: No Python thread is ever fork safe because the Python interpreter itself can never be made fork safe. Nor should anyone try to make the interpreter itself safe. It is too complex

[issue6721] Locks in python standard library should be sanitized on fork

2011-07-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: If Nir's analysis is right, and Antoines comment pushes me into this direction, (i personally have not looked at that code), then multiprocessing is completely brain-damaged and has been implemented by a moron. And yes, I know this

[issue6721] Locks in python standard library should be sanitized on fork

2011-07-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: Um, and just to add: i'm not watching out for anything, and it won't and it can't be me: ?0%0[steffen@sherwood sys]$ grep -F smp CHANGELOG.svn -B3 | grep -E '^r[[:digit:]]+' | tail -n 1 r162 | steffen | 2006-01-18 18:29:58 +0100

[issue6721] Locks in python standard library should be sanitized on fork

2011-07-19 Thread Nir Aides
Nir Aides n...@winpdb.org added the comment: then multiprocessing is completely brain-damaged and has been implemented by a moron. Please do not use this kind of language. Being disrespectful to other people hurts the discussion. -- ___ Python

[issue6721] Locks in python standard library should be sanitized on fork

2011-07-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: P.S.: I have to apologize, it's Tomaž, not Thomas. (And unless i'm mistaken this is pronounced TomAsch rather than the english Tommes, so i was just plain wrong.) --Steffen Ciao, sdaoden(*)(gmail.com) () ascii ribbon campaign -

[issue6721] Locks in python standard library should be sanitized on fork

2011-07-19 Thread Steffen Daode Nurpmeso
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment: then multiprocessing is completely brain-damaged and has been implemented by a moron. Please do not use this kind of language. Being disrespectful to other people hurts the discussion. So i apologize once again. 'Still i

  1   2   >