We were using the eventlet db pool class. You can see the code here:
https://github.com/openstack/nova/blob/2011.3/nova/db/sqlalchemy/session.py there were a few bugs about it, but here is one: https://bugs.launchpad.net/nova/+bug/838581 Vish On Mar 1, 2012, at 6:11 PM, Kapil Thangavelu wrote: > > I actually didn't mean to send the parent.. but since we're here. > > Additional threads/pool make the problem potentially worse, they are the > origin > of the greenlet switching issue that was the start of this thread. ie. with > the > monkey patching, most stdlib socket, thread module usage by python code in > the > non-main threads will potentially attempt a greenlet trampoline across > threads > at worst (ie. error), or unintentional per thread hubs at best. > > Since the mysqldb is an extension (no python code) wrapping it in eventlet > thread pool should work in theory. What where the problems with it last time > it > was attempted? > > cheers, > > -kapil > > > Excerpts from Devin Carlen's message of 2012-03-01 20:38:20 -0500: >> As long as we allocate a thread in the eventlet thread pool for the number >> of mysql connections we want to actually maintain in our connection pool, we >> shouldn't have problems getting the results we want even with the blocking >> mysql c drivers. >> >> Devin >> >> On Thursday, March 1, 2012 at 5:23 PM, Kapil Thangavelu wrote: >> >>> The standard python postgresql driver (psycopg2) does have an async mode. >>> There >>> are non db api compliant async mysql drivers for gevent. >>> >>> >>> Excerpts from Vishvananda Ishaya's message of 2012-03-01 15:36:43 -0500: >>>> Yes it does. We actually tried to use a pool at diablo release and it was >>>> very broken. There was discussion about moving over to a pure-python mysql >>>> library, but it hasn't been tried yet. >>>> >>>> Vish >>>> >>>> On Mar 1, 2012, at 11:45 AM, Yun Mao wrote: >>>> >>>>> There are plenty eventlet discussion recently but I'll stick my >>>>> question to this thread, although it's pretty much a separate >>>>> question. :) >>>>> >>>>> How is MySQL access handled in eventlet? Presumably it's external C >>>>> library so it's not going to be monkey patched. Does that make every >>>>> db access call a blocking call? Thanks, >>>>> >>>>> Yun >>>>> >>>>> On Wed, Feb 29, 2012 at 9:18 PM, Johannes Erdfelt <johan...@erdfelt.com >>>>> (mailto:johan...@erdfelt.com)> wrote: >>>>>> On Wed, Feb 29, 2012, Yun Mao <yun...@gmail.com >>>>>> (mailto:yun...@gmail.com)> wrote: >>>>>>> Thanks for the explanation. Let me see if I understand this. >>>>>>> >>>>>>> 1. Eventlet will never have this problem if there is only 1 OS thread >>>>>>> -- let's call it main thread. >>>>>>> >>>>>> >>>>>> >>>>>> In fact, that's exactly what Python calls it :) >>>>>> >>>>>>> 2. In Nova, there is only 1 OS thread unless you use xenapi and/or the >>>>>>> virt/firewall driver. >>>>>>> 3. The python logging module uses locks. Because of the monkey patch, >>>>>>> those locks are actually eventlet or "green" locks and may trigger a >>>>>>> green thread context switch. >>>>>>> >>>>>>> Based on 1-3, does it make sense to say that in the other OS threads >>>>>>> (i.e. not main thread), if logging (plus other pure python library >>>>>>> code involving locking) is never used, and we do not run a eventlet >>>>>>> hub at all, we should never see this problem? >>>>>>> >>>>>> >>>>>> >>>>>> That should be correct. I'd have to double check all of the monkey >>>>>> patching that eventlet does to make sure there aren't other cases where >>>>>> you may inadvertently use eventlet primitives across real threads. >>>>>> >>>>>> JE >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Mailing list: https://launchpad.net/~openstack >>>>>> Post to : openstack@lists.launchpad.net >>>>>> (mailto:openstack@lists.launchpad.net) >>>>>> Unsubscribe : https://launchpad.net/~openstack >>>>>> More help : https://help.launchpad.net/ListHelp >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mailing list: https://launchpad.net/~openstack >>>>> Post to : openstack@lists.launchpad.net >>>>> (mailto:openstack@lists.launchpad.net) >>>>> Unsubscribe : https://launchpad.net/~openstack >>>>> More help : https://help.launchpad.net/ListHelp >>>>> >>>> >>>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : openstack@lists.launchpad.net >>> (mailto:openstack@lists.launchpad.net) >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp >>> >>>
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
