That would be a method of what class? On Feb 25, 2009, at 10:43 AM, Aron Bierbaum wrote:
> I guess I don't see where these discussions would effect what we are > doing. I am sure I am missing something, but in order to allow no > changes by users, can't we just do something like: > > def __call__(self, *args): > if not hasattr(thread_data, 'geos_handle'): > thread_data.geos_handle = > lgeos._lgeos.initGEOS_r(notice_h, error_h) > > -Aron > > On Wed, Feb 25, 2009 at 10:26 AM, Sean Gillies <[email protected]> > wrote: > Aron, > > r1194 is indeed the last stable revision for a while. > > Importing modules from a thread is fraught with all kinds of issues. > See this interesting discussion: > > http://bugs.python.org/issue1720705 > > And the conclusion at the end of > > http://svn.python.org/view/python/trunk/Doc/library/threading.rst?view=markup&pathrev=61365 > > I think we're going to need a module level function to initialize > thread local data. > > On Feb 24, 2009, at 5:23 PM, Aron Bierbaum wrote: > > > Well I finally got around to testing these changes. It appears that > > there are a lot of changes being made currently, which revision > should > > I be testing with? I have tried running test_threads.py with both > > trunk and r1194 and the test fails. I briefly looked at r1194 and it > > looks like the thread specific geos_handle is not getting created in > > each thread. It is created correctly in the primordially thread, but > > not the others. I believe it is because the threading.local > > constructor is only called once, not once per thread. Any ideas on a > > fix? > > > > Thanks, > > Aron > > > > On Thu, Jan 29, 2009 at 5:59 PM, Aron Bierbaum > > <[email protected]> wrote: > >> Wow, this looks really nice. I should have time to test it either > >> later tonight or tomorrow. It's also interesting that this should > >> allow an easy transition if/when they move to a thread safe only > >> interface. > >> > >> Thanks, > >> Aron > >> > >> On Thu, Jan 29, 2009 at 4:33 PM, Sean Gillies <[email protected]> > >> wrote: > >>> Aron, > >>> > >>> I did a bit of work along those lines and made some pretty good > >>> progress. Tests are passing except for a couple I had to disable. > >>> Updating of coordinate sequences through geos_linestring and > >>> geos_linearring is currently broken. I'm not sure whether it's my > >>> fault or a bug in the reentrant API. Probably the former. > >>> > >>> Checkout r1194 if you're interested in testing. I haven't tried it > >>> in > >>> a threaded situation yet, and suspect it will be pretty hard to > >>> trigger a fault -- Shapely itself doesn't expose any means of > >>> modifying the GEOS context data, and I don't see it being done > >>> inside > >>> GEOS. > >>> > >>> Cheers, > >>> Sean > >>> > >>> On Jan 28, 2009, at 10:44 PM, Aron Bierbaum wrote: > >>> > >>>> This is a really long shot but what do people think of something > >>>> like > >>>> the following. I have run into a few problems with ctypes while > >>>> trying > >>>> to run with it, but I thought it might start people thinking. > >>>> > >>>> import threading > >>>> import functools > >>>> > >>>> class LocalStorage(threading.local): > >>>> def __getattr__(self, name): > >>>> if "handle" == name: > >>>> self.handle = geos_lib.initGEOS_r(notice_h, error_h) > >>>> #print "Construct handle:", > threading.currentThread().name > >>>> #print " id:", id(self.handle) > >>>> return self.handle > >>>> raise AttributeError("Local storage does not have attribute > >>>> '%s'" % name) > >>>> > >>>> local_storage = LocalStorage() > >>>> > >>>> class ThreadSafeGeos(object): > >>>> def __getattr__(self, name): > >>>> new_name = "%s_r" % (name) > >>>> r_func = getattr(geos_lib, new_name) > >>>> handle = local_storage.handle > >>>> return functools.partial(r_func, handle) > >>>> > >>>> tsgeos = ThreadSafeGeos() > >>>> > >>>> lgeos = tsgeos > >>>> > >>>> > >>>> -Aron > >>>> > >>>> > >>>> On Tue, Jan 27, 2009 at 8:16 AM, Justin Bronn <[email protected]> > >>>> wrote: > >>>>> Aron Bierbaum wrote: > >>>>>> The only idea that I have off the top of my head is to use > >>>>>> threading.local in order to keep track of a handle for each > >>>>>> thread > >>>>>> that calls any function in GEOS. I tried implementing something > >>>>>> like > >>>>>> this quickly and ran into problems. Any other ideas? > >>>>> > >>>>> Since I'm going to have to do the same for GeoDjango's bindings, > >>>>> I've > >>>>> been kicking around some ideas on how to do this -- and I also > >>>>> thought > >>>>> of something similar. I think this idea would work, but it's > >>>>> going > >>>>> to > >>>>> require some extensive implementation. > >>>>> > >>>>> Basically you'd have a global dict keyed by > >>>>> threading.currentThread() > >>>>> and a corresponding value be the GEOS context handle. However, > >>>>> you'd > >>>>> have to (have a way to determine whether you're running GEOS 3.1 > >>>>> or > >>>>> not > >>>>> and use the appropriate threading/non-threading function > >>>>> signature -- > >>>>> including dynamically insert the context handle into the GEOS C > >>>>> API > >>>>> function arguments. I'm wondering how much overhead this is and > >>>>> what > >>>>> the effect on performance would be -- but I haven't had time > (and > >>>>> won't > >>>>> for the next several weeks) to mock up and try it out. > >>>>> > >>>>>>> Thread safety hasn't been a problem up to now, and we've > >>>>>>> hammered > >>>>>>> on > >>>>>>> it quite a bit. We're using PyDLL to load libgeos_c, which > means > >>>>>>> the > >>>>>>> GIL isn't released. > >>>>>>> > >>>>> > >>>>> Yeah, it's a problem that's not seen until two routines enter > the > >>>>> same > >>>>> critical areas in GEOS at the same time. For web apps the > problem > >>>>> (segfaulting Apache/FCGI processes) becomes more apparent as > >>>>> traffic > >>>>> increases for the site. I consider it a serious problem. > >>>>> > >>>>> -Justin > >>>>> _______________________________________________ > >>>>> Community mailing list > >>>>> [email protected] > >>>>> http://lists.gispython.org/mailman/listinfo/community > >>>>> > >>>> _______________________________________________ > >>>> Community mailing list > >>>> [email protected] > >>>> http://lists.gispython.org/mailman/listinfo/community > >>> > >>> _______________________________________________ > >>> Community mailing list > >>> [email protected] > >>> http://lists.gispython.org/mailman/listinfo/community > >>> > >> > > _______________________________________________ > > Community mailing list > > [email protected] > > http://lists.gispython.org/mailman/listinfo/community > > _______________________________________________ > Community mailing list > [email protected] > http://lists.gispython.org/mailman/listinfo/community > > _______________________________________________ > Community mailing list > [email protected] > http://lists.gispython.org/mailman/listinfo/community _______________________________________________ Community mailing list [email protected] http://lists.gispython.org/mailman/listinfo/community
