Aron, r1194 is indeed the last stable revision for a while.
Importing modules from a thread is fraught with all kinds of issues. See this interesting discussion: http://bugs.python.org/issue1720705 And the conclusion at the end of http://svn.python.org/view/python/trunk/Doc/library/threading.rst?view=markup&pathrev=61365 I think we're going to need a module level function to initialize thread local data. On Feb 24, 2009, at 5:23 PM, Aron Bierbaum wrote: > Well I finally got around to testing these changes. It appears that > there are a lot of changes being made currently, which revision should > I be testing with? I have tried running test_threads.py with both > trunk and r1194 and the test fails. I briefly looked at r1194 and it > looks like the thread specific geos_handle is not getting created in > each thread. It is created correctly in the primordially thread, but > not the others. I believe it is because the threading.local > constructor is only called once, not once per thread. Any ideas on a > fix? > > Thanks, > Aron > > On Thu, Jan 29, 2009 at 5:59 PM, Aron Bierbaum > <[email protected]> wrote: >> Wow, this looks really nice. I should have time to test it either >> later tonight or tomorrow. It's also interesting that this should >> allow an easy transition if/when they move to a thread safe only >> interface. >> >> Thanks, >> Aron >> >> On Thu, Jan 29, 2009 at 4:33 PM, Sean Gillies <[email protected]> >> wrote: >>> Aron, >>> >>> I did a bit of work along those lines and made some pretty good >>> progress. Tests are passing except for a couple I had to disable. >>> Updating of coordinate sequences through geos_linestring and >>> geos_linearring is currently broken. I'm not sure whether it's my >>> fault or a bug in the reentrant API. Probably the former. >>> >>> Checkout r1194 if you're interested in testing. I haven't tried it >>> in >>> a threaded situation yet, and suspect it will be pretty hard to >>> trigger a fault -- Shapely itself doesn't expose any means of >>> modifying the GEOS context data, and I don't see it being done >>> inside >>> GEOS. >>> >>> Cheers, >>> Sean >>> >>> On Jan 28, 2009, at 10:44 PM, Aron Bierbaum wrote: >>> >>>> This is a really long shot but what do people think of something >>>> like >>>> the following. I have run into a few problems with ctypes while >>>> trying >>>> to run with it, but I thought it might start people thinking. >>>> >>>> import threading >>>> import functools >>>> >>>> class LocalStorage(threading.local): >>>> def __getattr__(self, name): >>>> if "handle" == name: >>>> self.handle = geos_lib.initGEOS_r(notice_h, error_h) >>>> #print "Construct handle:", threading.currentThread().name >>>> #print " id:", id(self.handle) >>>> return self.handle >>>> raise AttributeError("Local storage does not have attribute >>>> '%s'" % name) >>>> >>>> local_storage = LocalStorage() >>>> >>>> class ThreadSafeGeos(object): >>>> def __getattr__(self, name): >>>> new_name = "%s_r" % (name) >>>> r_func = getattr(geos_lib, new_name) >>>> handle = local_storage.handle >>>> return functools.partial(r_func, handle) >>>> >>>> tsgeos = ThreadSafeGeos() >>>> >>>> lgeos = tsgeos >>>> >>>> >>>> -Aron >>>> >>>> >>>> On Tue, Jan 27, 2009 at 8:16 AM, Justin Bronn <[email protected]> >>>> wrote: >>>>> Aron Bierbaum wrote: >>>>>> The only idea that I have off the top of my head is to use >>>>>> threading.local in order to keep track of a handle for each >>>>>> thread >>>>>> that calls any function in GEOS. I tried implementing something >>>>>> like >>>>>> this quickly and ran into problems. Any other ideas? >>>>> >>>>> Since I'm going to have to do the same for GeoDjango's bindings, >>>>> I've >>>>> been kicking around some ideas on how to do this -- and I also >>>>> thought >>>>> of something similar. I think this idea would work, but it's >>>>> going >>>>> to >>>>> require some extensive implementation. >>>>> >>>>> Basically you'd have a global dict keyed by >>>>> threading.currentThread() >>>>> and a corresponding value be the GEOS context handle. However, >>>>> you'd >>>>> have to (have a way to determine whether you're running GEOS 3.1 >>>>> or >>>>> not >>>>> and use the appropriate threading/non-threading function >>>>> signature -- >>>>> including dynamically insert the context handle into the GEOS C >>>>> API >>>>> function arguments. I'm wondering how much overhead this is and >>>>> what >>>>> the effect on performance would be -- but I haven't had time (and >>>>> won't >>>>> for the next several weeks) to mock up and try it out. >>>>> >>>>>>> Thread safety hasn't been a problem up to now, and we've >>>>>>> hammered >>>>>>> on >>>>>>> it quite a bit. We're using PyDLL to load libgeos_c, which means >>>>>>> the >>>>>>> GIL isn't released. >>>>>>> >>>>> >>>>> Yeah, it's a problem that's not seen until two routines enter the >>>>> same >>>>> critical areas in GEOS at the same time. For web apps the problem >>>>> (segfaulting Apache/FCGI processes) becomes more apparent as >>>>> traffic >>>>> increases for the site. I consider it a serious problem. >>>>> >>>>> -Justin >>>>> _______________________________________________ >>>>> Community mailing list >>>>> [email protected] >>>>> http://lists.gispython.org/mailman/listinfo/community >>>>> >>>> _______________________________________________ >>>> Community mailing list >>>> [email protected] >>>> http://lists.gispython.org/mailman/listinfo/community >>> >>> _______________________________________________ >>> Community mailing list >>> [email protected] >>> http://lists.gispython.org/mailman/listinfo/community >>> >> > _______________________________________________ > Community mailing list > [email protected] > http://lists.gispython.org/mailman/listinfo/community _______________________________________________ Community mailing list [email protected] http://lists.gispython.org/mailman/listinfo/community
