On Mon, Aug 31, 2009 at 2:55 PM, Benjamin Ducke<[email protected]> wrote: > Dear all, > > After my recent experiments with v.out.ogr, I noticed a number > of weaknesses that show particularly when working with data > stored in a DBMS. > > I would like to put some work into improving it. > So could I have your comments on the following thoughts, > please? > > 1. The GRASS primary key "cat" column always gets exported > (and upon re-import and export, you may get "_cat", ...). > This does not always seem to be useful and can even be > significant bloat for very large datasets. How about adding > a flag to drop (or preserve?) it in the output?
A flag to deactivate it would be nice. Like this the module keeps common behavior and the flag is an addon. > 2. There is currently no implemented action for exporting > "Kernel" type geometries. Should they be exported simply > as 3D points? For the time being perhaps yes (no idea if there are any OGC specs for this or plans in OGR) > 3. Export in general is _very_ slow (I think this has been > discussed here before). On my machine v.out.ogr takes almost > 8 mins to export 32,000 points to an SQLite database file. I was hit by the same problem today: Placenames of Italy # in LatLong location: wget -chttp://download.geonames.org/export/dump/IT.zip unzip IT.zip IT.txt v.in.geonames in=IT.txt out=italy_geonames v.out.ogr italy_geonames dsn=italy_geonames_LL.shp type=point ... this takes forever. But: > Compare that to ogr2ogr which does the same job (plus importing > the input data) in 4 seconds! Right: ogr2ogr italy_geonames_LL.shp grassdata/latlong_wgs84/neteler/vector/italy_geonames/head ... perhaps 10 seconds for 43000 points. Since ogr2ogr uses the GRASS plugin to read the data, the problem must be in the GRASS part of v.out.ogr. > All that extra time seems to > be spent in mk_at() which generates the attribute table. > The DBMS management code look like quite some overhead. > How about loading all the attribute tables into a simple > in-mem C structure in one go and dumping them straight into > each OGR feature definition on export? This would mean that > the whole attribute table needs to be in memory (but it > should also be possible to do this in segments) and could > be controlled via a module flag. Or is that unlikely to > gain much performance? No idea but certainly worth trying. How does ogr2ogr deal with it? > 4. SQLite happens to be an OGR driver that does not supports > overwriting existing datasources. Thus, > > Ogr_ds = OGR_Dr_CreateDataSource(Ogr_driver, dsn_opt->answer, > papszDSCO); > > as used by v.in.ogr will always fail. Worth an OGR bug ticket? > This means that users > are confined to exporting data to a new table in a new SQLite > database only. In fact, SQLite is capable of updating an existing > database, by adding a new table to it. How about a new flag "-u" > to open an OGR datasource in "update" mode, using OGR_Dr_Open(?) > This could also benefit other OGR drivers with similar design > issues. Sounds good! Markus _______________________________________________ grass-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-dev
