Wow, this looks really nice. I should have time to test it either later tonight or tomorrow. It's also interesting that this should allow an easy transition if/when they move to a thread safe only interface.
Thanks, Aron On Thu, Jan 29, 2009 at 4:33 PM, Sean Gillies <[email protected]> wrote: > Aron, > > I did a bit of work along those lines and made some pretty good > progress. Tests are passing except for a couple I had to disable. > Updating of coordinate sequences through geos_linestring and > geos_linearring is currently broken. I'm not sure whether it's my > fault or a bug in the reentrant API. Probably the former. > > Checkout r1194 if you're interested in testing. I haven't tried it in > a threaded situation yet, and suspect it will be pretty hard to > trigger a fault -- Shapely itself doesn't expose any means of > modifying the GEOS context data, and I don't see it being done inside > GEOS. > > Cheers, > Sean > > On Jan 28, 2009, at 10:44 PM, Aron Bierbaum wrote: > >> This is a really long shot but what do people think of something like >> the following. I have run into a few problems with ctypes while trying >> to run with it, but I thought it might start people thinking. >> >> import threading >> import functools >> >> class LocalStorage(threading.local): >> def __getattr__(self, name): >> if "handle" == name: >> self.handle = geos_lib.initGEOS_r(notice_h, error_h) >> #print "Construct handle:", threading.currentThread().name >> #print " id:", id(self.handle) >> return self.handle >> raise AttributeError("Local storage does not have attribute >> '%s'" % name) >> >> local_storage = LocalStorage() >> >> class ThreadSafeGeos(object): >> def __getattr__(self, name): >> new_name = "%s_r" % (name) >> r_func = getattr(geos_lib, new_name) >> handle = local_storage.handle >> return functools.partial(r_func, handle) >> >> tsgeos = ThreadSafeGeos() >> >> lgeos = tsgeos >> >> >> -Aron >> >> >> On Tue, Jan 27, 2009 at 8:16 AM, Justin Bronn <[email protected]> >> wrote: >>> Aron Bierbaum wrote: >>>> The only idea that I have off the top of my head is to use >>>> threading.local in order to keep track of a handle for each thread >>>> that calls any function in GEOS. I tried implementing something like >>>> this quickly and ran into problems. Any other ideas? >>> >>> Since I'm going to have to do the same for GeoDjango's bindings, I've >>> been kicking around some ideas on how to do this -- and I also >>> thought >>> of something similar. I think this idea would work, but it's going >>> to >>> require some extensive implementation. >>> >>> Basically you'd have a global dict keyed by threading.currentThread() >>> and a corresponding value be the GEOS context handle. However, you'd >>> have to (have a way to determine whether you're running GEOS 3.1 or >>> not >>> and use the appropriate threading/non-threading function signature -- >>> including dynamically insert the context handle into the GEOS C API >>> function arguments. I'm wondering how much overhead this is and what >>> the effect on performance would be -- but I haven't had time (and >>> won't >>> for the next several weeks) to mock up and try it out. >>> >>>>> Thread safety hasn't been a problem up to now, and we've hammered >>>>> on >>>>> it quite a bit. We're using PyDLL to load libgeos_c, which means >>>>> the >>>>> GIL isn't released. >>>>> >>> >>> Yeah, it's a problem that's not seen until two routines enter the >>> same >>> critical areas in GEOS at the same time. For web apps the problem >>> (segfaulting Apache/FCGI processes) becomes more apparent as traffic >>> increases for the site. I consider it a serious problem. >>> >>> -Justin >>> _______________________________________________ >>> Community mailing list >>> [email protected] >>> http://lists.gispython.org/mailman/listinfo/community >>> >> _______________________________________________ >> Community mailing list >> [email protected] >> http://lists.gispython.org/mailman/listinfo/community > > _______________________________________________ > Community mailing list > [email protected] > http://lists.gispython.org/mailman/listinfo/community > _______________________________________________ Community mailing list [email protected] http://lists.gispython.org/mailman/listinfo/community
