Hi Sean, 2009/4/4 Sean Gillies <[email protected]>
> Hi Pascal, > > This cries out for a cascaded union, I think. Maybe we should have a > 1.0.12 release. Still, the leaks trouble me. Statements like > > P = P.union(p) > > have caused me problems in the past. Probably a matter of held > references? Any interest in digging into it? > Of course I'm interested in finding what's going wrong. But I've using Python (and Shapely) for less than a year (my first mail to the mailing list was posted in last July) and I think my knowledge in Python is not big enough, especially for Python/C interface, to fix the issue. But I'm trying to get into your code and learn how Shapely works. I would be so sad to have to work again with C/libgeos. Once again, Shapely saves me time (thanks !) and I've taught my colleagues (Institut Géographique National, France) to use Python and Shapely . If I find something, I tell you. Pascal > > Sean > > On Apr 3, 2009, at 3:02 AM, Pascal Leroux wrote: > > > Even with very simple geometries : > > > > >>> P = Polygon(((0,0),(0,9),(9,9),(9,0))) > > >>> p = Polygon(((1,1),(1,5),(5,5),(5,1))) > > >>> count = 1 > > >>> while 1: > > ... if count % 10000 == 0: print count > > ... P = P.union(p) > > ... count += 1 > > > > > > after 25 minutes : > > > > > > 7020000 > > 7030000 > > Python(4969) malloc: *** mmap(size=2097152) failed (error code=12) > > *** error: can't allocate region > > *** set a breakpoint in malloc_error_break to debug > > ... > > Python(4969) malloc: *** mmap(size=2097152) failed (error code=12) > > *** error: can't allocate region > > *** set a breakpoint in malloc_error_break to debug > > MemoryError > > Python(4969) malloc: *** mmap(size=2097152) failed (error code=12) > > *** error: can't allocate region > > *** set a breakpoint in malloc_error_break to debug > > Traceback (most recent call last): > > File "<stdin>", line 3, in <module> > > File "/System/Library/Frameworks/Python.framework/Versions/2.5/lib/ > > python2.5/site-packages/Shapely-1.0.11-py2.5.egg/shapely/ > > topology.py", line 34, in __call__ > > if not self.context.is_valid: > > MemoryError > > >>> > > > > > > This code doesn't make sense but it highlights the issue I talked > > about last week. > > I used a very big polygon in my last email but, here, I cannot use > > simpler ones. > > > > My final purpose is to build a full topology with several shapefiles > > located on a 1°x1° area and I often work with more than 200,000 > > features at a time. Their geometries, even if they are not as > > complicated as the polygon I used last week, are not as simple as > > those I used above. > > > > 7,000,000 calls to union with the two simple polygons crashed the > > script because of a lack of memory. That corresponds (in the better > > case), with 200,000 features, to an average of 35 calls per initial > > feature. Obviously, it's not enough to build a full topology. > > > > I'm trying to split my area into several parts, to build topology > > for each part and then merge the parts. It will be more complicated. > > Will it be enough ? Will I have to come back to C/libgeos ? aaargh ! > > > > Pascal > > > > > > > > _______________________________________________ > > Community mailing list > > [email protected] > > http://lists.gispython.org/mailman/listinfo/community > > _______________________________________________ > Community mailing list > [email protected] > http://lists.gispython.org/mailman/listinfo/community >
_______________________________________________ Community mailing list [email protected] http://lists.gispython.org/mailman/listinfo/community
