Hi,
Just a quick stupid question: are you writing to a local file system or
to a network filesystem. The latter can be substantially slower than the
former, esp. with Geopackage.
Andreas
Am 01.11.18 um 12:10 schrieb koji higuchi:
Hi Even
I managed to write gpkg without ogr without fiona using class
ShapeConverter(osmium.SimpleHandler).
still, speed seems slow.
Any idea to increase it up.
On Thu, Nov 1, 2018 at 7:05 PM koji higuchi <[email protected]
<mailto:[email protected]>> wrote:
Hi Even
I am trying to replace using fiona by pure ogr.
but having difficult to collapse the code with class and objects.
If i could extract geom and tags in a ogr pythonic way, then it be
better
On Thu, Nov 1, 2018 at 6:45 PM Even Rouault
<[email protected] <mailto:[email protected]>>
wrote:
You are perhaps using a too old version of Fiona. Recent
versions insert
features within OGR transactions. Should be fine with latest
Fiona 1.8.0
See https://github.com/Toblerity/Fiona/issues/476
> Hi, following is the code I used:
>
> Import os, osmium, fiona
>
> fi = 'europe-latest.osm.pbf'
> fo = 'europe-latest.dpkg'
>
> drv = 'DPKG'
>
> crs = {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84',
'proj':
> 'longlat'}
>
> schema = {'geometry': 'LineString',
> 'properties': {'id': 'float', 'name' :
'str', 'kind' :
> 'str'}}
>
> outfile = fiona.open(fo, 'w', driver=drv, crs=crs,
schema=schema)
>
> geomfab = osmium.geom.GeoJSONFactory()
>
> class ShapeConverter(osmium.SimpleHandler):
> def way(self, w):
> if 'place' in w.tags:
> rec = {'geometry' :
eval(geomfab.create_linestring(w)),
> 'properties' : {'id' : float(w.id
<http://w.id>),
> 'name' :
w.tags.get('name'),
> 'kind' :
w.tags['place']}}
> outfile.write(rec)
>
> ShapeConverter().apply_file(fi, locations=True)
>
> On Thu, Nov 1, 2018 at 4:10 PM jratike80 <
>
> [email protected]
<mailto:[email protected]>> wrote:
> > koji higuchi wrote
> >
> > > Hi
> > > I am extracting .osm.pbf file into .gpkg; but the
writing rate is very
> > > slow.
> > > When I write into .shp, rate is faster but its limit is 4gb.
> > > So, what other format is better for larger than 4gb
requirement?
> > > Thanks for your ideas.
> > > Koji
> >
> > Hi,
> >
> > Show the exact command that you are using so we can see if
it could be
> > made
> > faster. For example if you happen to use -skipfailures for
handling the
> > invalid geometries of the OSM data then you will make GDAL
to do a new
> > transaction for each row and that certainly is slow. But
let's have a look
> > at your command first.
> >
> > -Jukka Rahkonen-
> >
> >
> >
> > --
> > Sent from:
http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
> > _______________________________________________
> > gdal-dev mailing list
> > [email protected] <mailto:[email protected]>
> > https://lists.osgeo.org/mailman/listinfo/gdal-dev
--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________
gdal-dev mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
