HI, I tried again with the koordinates shapefile. I did succeed that time - corrupted shapefile I suppose.
Thanks for the help, Pierre 2010/11/30 Markus Metz <[email protected]>: > Pierre Roudier wrote: >> Dear all, >> >> I tried the very same thing on the stable version of GRASS (6.4 RC7 - >> which AFAIK is the stable 6.4), on the very same machine. > > The stable 6.4 version is 6.4.0, the recommended 6.4 version is 6.4.1 > >> Unfortunately, I still an error. On the "Break boundaries" state >> during the imprt, it is stuck for a while at 4gb RAM used. Then for >> some reason the memory consumption becomes much bigger (maximum >> actually - 6.2GB+2Gb swap) and the process gets killed. >> > Weird. I assume you want to import the Land Cover DataBase version 2 > for the Northern Island of New Zealand in the New Zealand Map Grid > projection. I have imported the Land Cover DataBase version 2 for all > of New Zealand in the New Zealand Transverse Mercator projection with > more than twice as many polygons as in the Northern Island only, and > memory peak was at about 4GB. Judging by the cleaning done by v.in.ogr > --v, the shapefile I imported was clean. For not so clean shapefiles > where adjacent polygons are not 100% adjacent but are slightly > overlapping or leave small gaps in between them, memory consumption > can go up considerably. > > I have two suggestions: > 1. Use the snapping option of v.in.ogr with snap=1 and have verbose > output with --v. Considering that the LCDB2 was created from Landsat > ETM7+ imagery, the spatial detail can not be finer than 28.5m and you > may be able to increase the snapping threshold to values larger than 1 > without snapping away polygons. > > 2. Use the lcdb2 version I used, available at koordinates.com [1]. > Optionally set the region first to the Northern Island and then import > with v.in.ogr -r > >> Any pointers would be much appreciated. Otherwise, maybe I should open >> a bug for that? >> > I don't think this is a bug. Memory consumption can get quite high > when processing larger vectors with topology in grass. I know which > are the memory-hungry components and want to reduce memory > consumption, but that will happen only in grass 7, and most probably > not this year. > > Markus M > > [1] http://koordinates.com/layer/1072-land-cover-database-version-2-lcdb2/ > >> >> 2010/11/28 Pierre Roudier <[email protected]>: >>> Thanks for your answers. Yes, this is a 64bits OS. >>> >>> 2010/11/27 Markus Metz <[email protected]>: >>>> On Fri, Nov 26, 2010 at 3:14 AM, Pierre Roudier >>>> <[email protected]> wrote: >>>>> Dear list, >>>>> >>>>> I am trying to load a pretty big shapefile on GRASS (6Mb +, 211621 >>>>> features) using v.in.ogr, but each attempt fails at the 'break >>>>> boundaries'' stage (at 98%!): >>>>> >>>>> GRASS 7.0.svn (NZTM2000):~ > v.in.ogr >>>>> dsn=~/Documents/DATA/LCDB2/ni_nzmg.shp out=ni_lcdb2 --o >>>>> Projection of input dataset and current location appear to match >>>>> Layer: ni_nzmg >>>>> Counting polygons for 211621 features... >>>>> Importing map 211621 features... >>>>> 100% >>>>> ----------------------------------------------------- >>>>> Building topology for vector map <ni_lcdb2_...@ni>... >>>>> Registering primitives... >>>>> 334611 primitives registered >>>>> 18459167 vertices registered >>>>> Number of nodes: 211681 >>>>> Number of primitives: 334611 >>>>> Number of points: 0 >>>>> Number of lines: 0 >>>>> Number of boundaries: 334611 >>>>> Number of centroids: 0 >>>>> Number of areas: - >>>>> Number of isles: - >>>>> ----------------------------------------------------- >>>>> WARNING: Cleaning polygons, result is not guaranteed! >>>>> ----------------------------------------------------- >>>>> Break polygons: >>>>> 100% >>>>> 100% >>>>> ----------------------------------------------------- >>>>> Remove duplicates: >>>>> 100% >>>>> ----------------------------------------------------- >>>>> Break boundaries: >>>>> ERROR: G_calloc: unable to allocate 50 * 8 bytes of memory at >>>>> allocation.c:82 >>>>> >>>>> The shp loads without any problems on QGIS. >>>>> >>>> >>>>> OS: Opensuse 11.3, with 7Gb RAM. >>>> Is this a 64 bit OS? >>>> >>>> I have previously successfully imported larger shapefiles, more than >>>> double in size: >400,000 areas, >40 million vertices, peak memory >>>> consumption was close to 4GB. The import of your shapefile would >>>> approximately require about 2GB of memory, which should not be a >>>> problem with 7GB available, as long as it's all 64 bit and not 32 bit. >>>> >>>> Markus M >>>> >>> >> > _______________________________________________ grass-user mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-user
