Even with very simple geometries :
>>> P = Polygon(((0,0),(0,9),(9,9),(9,0)))
>>> p = Polygon(((1,1),(1,5),(5,5),(5,1)))
>>> count = 1
>>> while 1:
... if count % 10000 == 0: print count
... P = P.union(p)
... count += 1
after 25 minutes :
7020000
7030000
Python(4969) malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
...
Python(4969) malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
MemoryError
Python(4969) malloc: *** mmap(size=2097152) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
Traceback (most recent call last):
File "<stdin>", line 3, in <module>
File
"/System/Library/Frameworks/Python.framework/Versions/2.5/lib/python2.5/site-packages/Shapely-1.0.11-py2.5.egg/shapely/topology.py",
line 34, in __call__
if not self.context.is_valid:
MemoryError
>>>
This code doesn't make sense but it highlights the issue I talked about last
week.
I used a very big polygon in my last email but, here, I cannot use simpler
ones.
My final purpose is to build a full topology with several shapefiles located
on a 1°x1° area and I often work with more than 200,000 features at a time.
Their geometries, even if they are not as complicated as the polygon I used
last week, are not as simple as those I used above.
7,000,000 calls to union with the two simple polygons crashed the script
because of a lack of memory. That corresponds (in the better case), with
200,000 features, to an average of 35 calls per initial feature. Obviously,
it's not enough to build a full topology.
I'm trying to split my area into several parts, to build topology for each
part and then merge the parts. It will be more complicated. Will it be
enough ? Will I have to come back to C/libgeos ? aaargh !
Pascal
_______________________________________________
Community mailing list
[email protected]
http://lists.gispython.org/mailman/listinfo/community
