I'm glad you like it, and thank you even more for the patch! I'll make a 1.0.14 release today.
Cheers, On Oct 6, 2009, at 3:14 AM, Paul Balomiri wrote: > Hi Sean, > > finally, the solution to the problem is attached to this mail. > > basically in ctypes SDLL/PyDLL loaders, the contained function > arguments and return value are not reflected. > By default when calling a function without supplying any argtypes list > and restype ctype it is assumed that the > respective argument can be casted to int. However, in case of c_void_p > this only works on > 32 bit platforms, where sizeof(c_size_t) = sizeof(c_int). All > pointers, such as c_void_p, > are of the same type as c_size_t. > > This produces to SEGFAULTs on 64 bit platforms where-ever specifying > argtypes and restype has been omitted. > > The bug only shows up when the pointer type > 2^32, otherwise python > converts the correct value to > c_void_p. On linux, and in my case at least it takes about 1500 > mallocs to hit one in the high memory area where pointers are > 2^32. > This might well be the reason why the problem has not shown up until > now. > > However, I had no time to thoroughly search for all imports used, but > not declared. I would be grateful if you could include the patch > into rev 1.0.13. I would have patched the trunk, but the file > structure there seems pretty different from the 1.0.13 version, and I > did't' find the > c-declarations python file. > > > Thank you for maintaining such a wonderful lib, > Cheers > > Paul > > 2009/10/5 Paul Balomiri <[email protected]>: >> Hi, >> >> I tried to find out where exactly BaseGeometry.__geom__ becomes int >> type. >> >> and here is what I found out: >> >>>>> p=Point(1.0,2.3) >> pointaddr=10373776 >>>>> p >> <shapely.geometry.point.Point object at 0x7f7f7e8ebed0> >>>>> p.__geom__.__class__ >> <type 'int'> >> >> >> OK, now going into the constructor of Point I could pinpoint >> Line #53 in point.py >> >> Creating a sequence there yielda an int instead of a c_void_p: >> >>>>> cs = lgeos.GEOSCoordSeq_create(1, 2) >>>>> cs.__class__ >> <type 'int'> >> >> This seems strance as in >> http://trac.gispython.org/lab/browser/Shapely/tags/rel-1.0.13/shapely/ctypes_declarations.py#L26 >> tou correctly defined >> >> lgeos.GEOSCoordSeq_create.restype = ctypes.c_void_p >> >> So the only thing coming to my mind is that the c_type_declarations >> might somehow not be included ? >> >> Do you have any hints where to go from here ? >> >> Cheers Paul >> >> 2009/10/5 Paul Balomiri <[email protected]>: >>> Hi, >>> Now having gone deeper into the matter i found out a bug, >>> and can further get into details about when a segfault still occurs. >>> The relevant fact i did not mention last time is that i'm running >>> shapely on a 64 bit CPU. >>> >>> 1) >>> http://trac.gispython.org/lab/browser/Shapely/tags/rel-1.0.13/shapely/geometry/base.py >>> ay line 352 you use >>> >>> size = c_int() >>> >>> and later on pass it as size size arg to GEOS' GEOSGeomToWKB_buf. >>> This is wrong on 64 bit processors. >>> Please use size = c_size_t() (8 byte long on 64 bit) >>> >>> 2) I could find what may be a corelated symptom, and easily to >>> check: >>> the __geom__ type is int. This causes segfaults when the actual >>> ponter type has >>> a value >2^32-1. I could not follow the code to the point where >>> __geom__ is set, >>> but am searching for it. Perhaps you can give me a hint ? >>> >>> >>> 2009/10/5 Sean Gillies <[email protected]>: >>>> On Oct 3, 2009, at 7:52 PM, Paul Balomiri wrote: >>>> >>>>> Hi, >>>>> >>>>> I encountered a similar problem to >>>>> http://lists.osgeo.org/pipermail/geos-devel/2009-July/ >>>>> 004280.html , >>>>> which is >>>>> at the exact same execution point. >>>>> >>>>> I am using shapely 1.0.13. As the lib is linked with PyDLL into >>>>> python, the GIL (Global Interpreter Log) is used to schedule >>>>> library >>>>> entrance ( = no python code (in any thread) runs concurrently), >>>>> so my first guess was, that an init error or an error in >>>>> handling the >>>>> global Context might cause the segfault. >>>>> >>>>> The backtrace I got from gdb is >>>>> >>>>> >>>>> [Switching to Thread 0x7fffe7f076f0 (LWP 6598)] >>>>> geos::io::WKBWriter::write (this=0x7fffeff120c0, g...@0xd8005270, >>>>> o...@0x7fffeff11f50) at WKBWriter.cpp:77 >>>>> 77 switch (g.getGeometryTypeId()) { >>>>> Current language: auto; currently c++ >>>>> (gdb) where >>>>> #0 geos::io::WKBWriter::write (this=0x7fffeff120c0, >>>>> g...@0xd8005270, >>>>> o...@0x7fffeff11f50) at WKBWriter.cpp:77 >>>>> #1 0x00007fffe6883549 in GEOSGeomToWKB_buf_r (extHandle=0xa5beb0, >>>>> g=0xd8005270, size=0x14407f0) at geos_ts_c.cpp:948 >>>>> #2 0x00007fffe6aa53b8 in ffi_call_unix64 () at >>>>> /build/buildd/python2.6-2.6.2/Modules/_ctypes/libffi/src/x86/ >>>>> unix64.S:75 >>>>> Python stacktrace: >>>>> >>>>> [Python Stack] >>>>> >>>>> #46 0x0000000000417ae9 in _start () at ../sysdeps/x86_64/elf/ >>>>> start.S: >>>>> 113 >>>>> >>>>> A full stacktrace is included as an attachment. I have augmented >>>>> the >>>>> stacktrace with the python >>>>> stackframe where the stack frame is in python code. >>>>> >>>>> I was wondering why GEOSGeomToWKB_buf is called, but >>>>> GEOSGeomToWKB_buf_r appears on the stacktrace. >>>>> >>>>> I do use threads in the application, but only the main thread has >>>>> calling points to shapely. >>>>> Also the application uses SQLAlchemy with the user defined >>>>> Geometry >>>>> type from shapely. The gluecode is part of mapfish. >>>>> >>>>> Perhaps you have some ideas about why the segfault shows up. I'm >>>>> not >>>>> familiar with the way >>>>> shapely handles GEOS c pointers. In gdb the geos geometry object >>>>> g...@0xd8005270 points to an adress >>>>> >>>>> trying to access the geometry object reveals >>>>> >>>>> (gdb) p g >>>>> $1 = (const geos::geom::Geometry &) @0xd8005270: <error reading >>>>> variable> >>>>> >>>>> >>>>> regards, >>>>> Paul >>>> >>>> Hi Paul, >>>> >>>> I'm sorry you're experiencing trouble with Shapely. If you can >>>> write a >>>> Python program that doesn't depend on MapFish or SQLAlchemy or >>>> PostGIS >>>> (I don't have that stack running right now) that reliably crashes, >>>> I'll get right on it. Maybe it's just a matter of writing out WKB >>>> from >>>> multiple threads at once. >>>> >>>> If you're using GEOS 3.1, the non-reentrant API is just a thin >>>> layer >>>> over the reentrant one, so EOSGeomToWKB_buf_r will always be >>>> called. >>>> >>>> Cheers, >>>> >>>> -- >>>> Sean >>>> >>>> _______________________________________________ >>>> Community mailing list >>>> [email protected] >>>> http://lists.gispython.org/mailman/listinfo/community >>>> >>> >>> >>> >>> -- >>> [email protected] >>> >> >> >> >> -- >> [email protected] >> > > > > -- > [email protected] > <Shapely.patch.rev1458>_______________________________________________ > Community mailing list > [email protected] > http://lists.gispython.org/mailman/listinfo/community -- Sean _______________________________________________ Community mailing list [email protected] http://lists.gispython.org/mailman/listinfo/community
