On Wed, Sep 9, 2009 at 20:19, Gregory P. Smith <g...@krypto.org> wrote:
> On Wed, Sep 9, 2009 at 9:07 AM, Thomas Wouters<tho...@python.org> wrote: > > > > > > On Sat, Jul 25, 2009 at 19:25, Gregory P. Smith <g...@krypto.org> wrote: > >> > >> On Thu, Jul 23, 2009 at 4:28 PM, Thomas Wouters<tho...@python.org> > wrote: > >> > > >> > So attached (and at http://codereview.appspot.com/96125/show ) is a > >> > preliminary fix, correcting the problem with os.fork(), os.forkpty() > and > >> > os.fork1(). This doesn't expose a general API for C code to use, for > two > >> > reasons: it's not easy, and I need this fix more than I need the API > >> > change > >> > :-) (I actually need this fix myself for Python 2.4, but it applies > >> > fairly > >> > cleanly.) > >> > >> This looks good to me. > > > > Anyone else want to take a look at this before I check it in? I updated > the > > patch (in Rietveld) to contain some documentation about the hazards of > > mixing fork and threads, which is the best we can do at the moment, at > least > > without seriously overhauling the threading APIs (which, granted, is not > > that bad an idea, considering the mess they're in.) I've now thoroughly > > tested the patch, and for most platforms it's strictly better. On AIX it > > *may* behave differently (possibly 'incorrectly' for specific cases) if > > something other than os.fork() calls the C fork() and calls > > PyOS_AfterFork(), since on AIX it used to nuke the thread lock. *I* think > > the new behaviour (not nuking the lock) is the correct thing to do, but > > since most places that release the import lock don't bother to check if > the > > lock was even held, the old behaviour may have been succesfully masking > the > > bug on AIX systems. > > Perhaps for the backport to 2.6 (which I think is in order, and also in > > accordance with the guidelines) I should leave the AIX workaround in? > Anyone > > think it should not be removed from 3.x/2.7? > > > >> > >> Your idea of making this an API called a 'fork lock' or something > >> sounds good (though I think it needs a better name. PyBeforeFork & > >> PyAfterFork?). The subprocess module, for example, disables garbage > >> collection before forking and restores it afterwards to avoid > >> http://bugs.python.org/issue1336. That type of thing could also be > >> done in such a function. > > > > Unfortunately it's rather hard to make those functions work correctly > with > > the current API -- we can't provide functions you can just use as > arguments > > to pthread_atfork because the global interpreter lock is not re-entrant > and > > we have no way of testing whether the current thread holds the GIL. I > also > > get the creepy-crawlies when I look at the various thread_* > implementations > > and see the horribly unsafe things they do (and also, for instance, the > > PendingCall stuff in ceval.c :S) Unfortunately there's no good way to fix > > these things without breaking API compatibility, let alone ABI > > compatibility. > > Take a look at http://code.google.com/p/python-atfork/ which I created > to address the general fork+threading with locks held causing > deadlocks issue with many standard library modules. Currently it only > patches the logging module but I intend to support others. I want to > get an atfork mechanism into 2.7/3.2 along with every lock in the > standard library making proper use of it. > > See also http://bugs.python.org/issue6721 > > I make no attempt to deal with C-level locks, only those acquired from > python. It doesn't use pthread_atfork but instead models its behavior > after that and wraps os.fork and os.forkpty so that they call the > registered atfork methods as appropriate. Well, sure, the *Python code* side of the problem is fixable, thanks to Python being so awesome ;-P I'm strictly thinking of applications embedding Python (or even extending it and calling into code that doesn't consider Python.) Your python-atfork project looks like a terribly good idea, but it won't fix the embedding/extending issues (nor is it intended to, right?) -- Thomas Wouters <tho...@python.org> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
