Seems my error was due to threading, specifically with the MySQL client. I had to alter the handler class so that it extended `threading.local` otherwise a MySQL query would block the whole process, causing error responses for other clients.
On Tue, Aug 3, 2010 at 2:41 PM, Phillip B Oldham <[email protected]> wrote: > Thanks Mark, that's really helpful. We're seeing a very odd bottleneck > and thought it could've been down to issues with threading in our > Python server. We've no shared state except what is persisted through > memcached and MySQL, so it mustn't be that. I'll continue to dig > around. > > Cheers! > > On Mon, Aug 2, 2010 at 8:57 PM, Mark Slee <[email protected]> wrote: >> Generally, no. Python's threading.local is a specific way of offering >> thread-local storage. >> >> You should NOT use threading.local if you're trying to safely share data >> between threads. That's not what it does -- rather it actually exposes a >> *different* copy of the data to each thread. >> >> So, you should use it if you actually want N copies of some data (where N is >> the number of threads), but you want a simple interface to refer to them all >> by the same name (with the clear understanding that you are actually >> accessing N different pieces of data). >> >> There are certainly situations in which this can be useful in a threaded >> Thrift server, but it's by no means a general rule. >> >> -----Original Message----- >> From: Phillip B Oldham [mailto:[email protected]] >> Sent: Friday, July 30, 2010 2:39 AM >> To: thrift-user >> Subject: Short question re. threading with a Python server >> >> Should Python classes that are run via the >> TThreaded/TThreadPool/TNonBlocking server inherit from >> `threading.local`, in order to be thread safe? >> >> -- >> Phillip B Oldham >> [email protected] >> +44 (0) 7525 01 09 01 >> > > > > -- > Phillip B Oldham > [email protected] > +44 (0) 7525 01 09 01 > -- Phillip B Oldham [email protected] +44 (0) 7525 01 09 01
