On 20/12/18 20:10, Markus Metz wrote:
On Thu, Dec 20, 2018 at 10:34 AM Markus Metz
<[email protected] <mailto:[email protected]>>
wrote:
>
>
>
> On Thu, Nov 29, 2018 at 3:43 PM Moritz Lennert
<[email protected] <mailto:[email protected]>> wrote:
> >
> > Hi,
> >
> > I have large amount of vector maps (~ 400) which each has hundreds of
> > thousands of areas and over 150 attribute columns. I'm looking for the
> > most efficient way to export these to a format I can share with
> > non-GRASS GIS users. My favourite would probably be Geopackage, amongst
> > others because the attribute column names are sometimes fairly long,
> > thus making export to a DBF based format not easily feasible.
> >
> > I've seen several sources of information about possible performance
> > increases, either in the SQLite world (as a Geopackage is based on
> > SQLite) or in the OGR SQLite driver page.
> >
> > For OGR [1], there is the -gt option to ogr2ogr which allows increasing
> > the number of features per transaction (or even running the entire
> > conversion in one single transaction).
[...]
> > How about the
> > transaction option ?
>
> I'm afraid the transaction option (equivalent to ogr2ogr -gt) is out
of control of v.out.ogr because it is format-specific and v.out.ogr does
not use the output format drivers directly, instead it uses the generic
OGR interface. However, we could try
GDALDatasetStartTransaction()/GDALDatasetCommitTransaction() in v.out.ogr.
v.out.ogr already uses the OGR interface to start and commit
transactions when possible.
Another reason for slow export can be a buggy NFS implementation if you
export to a NFS mount: check if v.out.ogr is running at 100%. If not you
could export to a local temp dir and then copy the GeoPackages to the
final destination.
Thanks for all the hints. Using
OGR_SQLITE_CACHE=1024 OGR_SQLITE_SYNCHRONOUS=OFF v.out.ogr ...
should already help.
In this particular case export was happening to local disk, so NFS was
not an issue.
Moritz
_______________________________________________
grass-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/grass-dev
