I'm writing file using class ShapeConverter(osmium.SimpleHandler) and ogr (not fiona) But, do not know whether transaction are all written instantaneously and simultaneously or not...
On Thu, Nov 1, 2018 at 8:34 PM koji higuchi <[email protected]> wrote: > Hi Andreas > I'm writing to a file into my PC disk. > > On Thu, Nov 1, 2018 at 8:32 PM Andreas Neumann <[email protected]> > wrote: > >> Hi, >> >> Just a quick stupid question: are you writing to a local file system or >> to a network filesystem. The latter can be substantially slower than the >> former, esp. with Geopackage. >> >> Andreas >> Am 01.11.18 um 12:10 schrieb koji higuchi: >> >> Hi Even >> I managed to write gpkg without ogr without fiona using class >> ShapeConverter(osmium.SimpleHandler). >> still, speed seems slow. >> Any idea to increase it up. >> >> On Thu, Nov 1, 2018 at 7:05 PM koji higuchi <[email protected]> >> wrote: >> >>> Hi Even >>> >>> I am trying to replace using fiona by pure ogr. >>> but having difficult to collapse the code with class and objects. >>> If i could extract geom and tags in a ogr pythonic way, then it be better >>> >>> >>> On Thu, Nov 1, 2018 at 6:45 PM Even Rouault <[email protected]> >>> wrote: >>> >>>> You are perhaps using a too old version of Fiona. Recent versions >>>> insert >>>> features within OGR transactions. Should be fine with latest Fiona 1.8.0 >>>> See https://github.com/Toblerity/Fiona/issues/476 >>>> >>>> > Hi, following is the code I used: >>>> > >>>> > Import os, osmium, fiona >>>> > >>>> > fi = 'europe-latest.osm.pbf' >>>> > fo = 'europe-latest.dpkg' >>>> > >>>> > drv = 'DPKG' >>>> > >>>> > crs = {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj': >>>> > 'longlat'} >>>> > >>>> > schema = {'geometry': 'LineString', >>>> > 'properties': {'id': 'float', 'name' : 'str', >>>> 'kind' : >>>> > 'str'}} >>>> > >>>> > outfile = fiona.open(fo, 'w', driver=drv, crs=crs, schema=schema) >>>> > >>>> > geomfab = osmium.geom.GeoJSONFactory() >>>> > >>>> > class ShapeConverter(osmium.SimpleHandler): >>>> > def way(self, w): >>>> > if 'place' in w.tags: >>>> > rec = {'geometry' : eval(geomfab.create_linestring(w)), >>>> > 'properties' : {'id' : float(w.id), >>>> > 'name' : w.tags.get('name'), >>>> > 'kind' : w.tags['place']}} >>>> > outfile.write(rec) >>>> > >>>> > ShapeConverter().apply_file(fi, locations=True) >>>> > >>>> > On Thu, Nov 1, 2018 at 4:10 PM jratike80 < >>>> > >>>> > [email protected]> wrote: >>>> > > koji higuchi wrote >>>> > > >>>> > > > Hi >>>> > > > I am extracting .osm.pbf file into .gpkg; but the writing rate is >>>> very >>>> > > > slow. >>>> > > > When I write into .shp, rate is faster but its limit is 4gb. >>>> > > > So, what other format is better for larger than 4gb requirement? >>>> > > > Thanks for your ideas. >>>> > > > Koji >>>> > > >>>> > > Hi, >>>> > > >>>> > > Show the exact command that you are using so we can see if it could >>>> be >>>> > > made >>>> > > faster. For example if you happen to use -skipfailures for handling >>>> the >>>> > > invalid geometries of the OSM data then you will make GDAL to do a >>>> new >>>> > > transaction for each row and that certainly is slow. But let's have >>>> a look >>>> > > at your command first. >>>> > > >>>> > > -Jukka Rahkonen- >>>> > > >>>> > > >>>> > > >>>> > > -- >>>> > > Sent from: >>>> http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html >>>> > > _______________________________________________ >>>> > > gdal-dev mailing list >>>> > > [email protected] >>>> > > https://lists.osgeo.org/mailman/listinfo/gdal-dev >>>> >>>> >>>> -- >>>> Spatialys - Geospatial professional services >>>> http://www.spatialys.com >>>> >>> >> _______________________________________________ >> gdal-dev mailing >> [email protected]https://lists.osgeo.org/mailman/listinfo/gdal-dev >> >> _______________________________________________ >> gdal-dev mailing list >> [email protected] >> https://lists.osgeo.org/mailman/listinfo/gdal-dev > >
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
