On Tue, 12 May 2015 at 05:08 Mike Bayer <mba...@redhat.com> wrote: > > On 5/11/15 2:02 PM, Attila Fazekas wrote: > > The scary part of a blocking I/O call is when you have two > > python thread (or green thread) and one of them is holding a DB lock the > other > > is waiting for the same lock in a native blocking I/O syscall. > that's a database deadlock and whether you use eventlet, threads, > asycnio or even just two transactions in a single-threaded script, that > can happen regardless. if your two eventlet "non blocking" greenlets > are waiting forever for a deadlock, you're just as deadlocked as if you > have OS threads. >
Not true (if I understand the situation Attila is referring to). > If you do a read(2) in native code, the python itself might not be able > to preempt it > > Your transaction might be finished with `DB Lock wait timeout`, > > with 30 sec of doing nothing, instead of scheduling to the another > python thread, > > which would be able to release the lock. > > > Here's the "you're losing me" part because Python threads are OS > threads, so Python isn't directly involved trying to "preempt" anything, > unless you're referring to the effect of the GIL locking up the > program. However, it's pretty easy to make two threads in Python hit a > database and do a deadlock against each other, and the rest of the > program's threads continue to run just fine; in a DB deadlock situation > you are blocked on IO and IO releases the GIL. > > If you can illustrate a test script that demonstrates the actual failing > of OS threads that does not occur greenlets here, that would make it > immediately apparent what it is you're getting at here. > 1. Thread A does something that takes a lock on the DB side 2. Thread B does something that blocks waiting for that same DB lock 3. Depends on the threading model - see below In a "true" preemptive threading system (eg: regular python threads), (3) is: 3. Eventually A finishes its transaction/whatever, commits and releases the DB lock 4. B then takes the lock and proceeds 5. Profit However, in a system where B's DB client can't be preempted (eg: eventlet or asyncio calling into a C-based mysql library, and A and B are running on the same underlying kernel thread), (3) is: 3. B will never be preempted, A will never be rescheduled, and thus A will never complete whatever it was doing. 4. Deadlock (in mysql-python's case, until a deadlock timer raises an exception and kills B 30s later) 5. Sadness. More specifically, we add a @retry to paper over the particular observed occurrence and then repeat this discussion on os-dev when the topic comes up again 6 months later. Note that this is not the usual database transaction deadlock caused by A and B each taking a lock and then trying to take the other's lock - this is a deadlock purely in the client-side code caused entirely by the lack of preemption during an otherwise safe series of DB operations. See my oslo.db unittest in Ib35c95defea8ace5b456af28801659f2ba67eb96 that reproduces the above with eventlet and allows you to test the behaviour of various DB drivers. (zzzeek: I know you've already seen all of the above in previous discussions, so sorry for repeating). - Gus
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev