Hi Andreas I'm writing to a file into my PC disk. On Thu, Nov 1, 2018 at 8:32 PM Andreas Neumann <[email protected]> wrote:
> Hi, > > Just a quick stupid question: are you writing to a local file system or to > a network filesystem. The latter can be substantially slower than the > former, esp. with Geopackage. > > Andreas > Am 01.11.18 um 12:10 schrieb koji higuchi: > > Hi Even > I managed to write gpkg without ogr without fiona using class > ShapeConverter(osmium.SimpleHandler). > still, speed seems slow. > Any idea to increase it up. > > On Thu, Nov 1, 2018 at 7:05 PM koji higuchi <[email protected]> wrote: > >> Hi Even >> >> I am trying to replace using fiona by pure ogr. >> but having difficult to collapse the code with class and objects. >> If i could extract geom and tags in a ogr pythonic way, then it be better >> >> >> On Thu, Nov 1, 2018 at 6:45 PM Even Rouault <[email protected]> >> wrote: >> >>> You are perhaps using a too old version of Fiona. Recent versions insert >>> features within OGR transactions. Should be fine with latest Fiona 1.8.0 >>> See https://github.com/Toblerity/Fiona/issues/476 >>> >>> > Hi, following is the code I used: >>> > >>> > Import os, osmium, fiona >>> > >>> > fi = 'europe-latest.osm.pbf' >>> > fo = 'europe-latest.dpkg' >>> > >>> > drv = 'DPKG' >>> > >>> > crs = {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj': >>> > 'longlat'} >>> > >>> > schema = {'geometry': 'LineString', >>> > 'properties': {'id': 'float', 'name' : 'str', >>> 'kind' : >>> > 'str'}} >>> > >>> > outfile = fiona.open(fo, 'w', driver=drv, crs=crs, schema=schema) >>> > >>> > geomfab = osmium.geom.GeoJSONFactory() >>> > >>> > class ShapeConverter(osmium.SimpleHandler): >>> > def way(self, w): >>> > if 'place' in w.tags: >>> > rec = {'geometry' : eval(geomfab.create_linestring(w)), >>> > 'properties' : {'id' : float(w.id), >>> > 'name' : w.tags.get('name'), >>> > 'kind' : w.tags['place']}} >>> > outfile.write(rec) >>> > >>> > ShapeConverter().apply_file(fi, locations=True) >>> > >>> > On Thu, Nov 1, 2018 at 4:10 PM jratike80 < >>> > >>> > [email protected]> wrote: >>> > > koji higuchi wrote >>> > > >>> > > > Hi >>> > > > I am extracting .osm.pbf file into .gpkg; but the writing rate is >>> very >>> > > > slow. >>> > > > When I write into .shp, rate is faster but its limit is 4gb. >>> > > > So, what other format is better for larger than 4gb requirement? >>> > > > Thanks for your ideas. >>> > > > Koji >>> > > >>> > > Hi, >>> > > >>> > > Show the exact command that you are using so we can see if it could >>> be >>> > > made >>> > > faster. For example if you happen to use -skipfailures for handling >>> the >>> > > invalid geometries of the OSM data then you will make GDAL to do a >>> new >>> > > transaction for each row and that certainly is slow. But let's have >>> a look >>> > > at your command first. >>> > > >>> > > -Jukka Rahkonen- >>> > > >>> > > >>> > > >>> > > -- >>> > > Sent from: >>> http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html >>> > > _______________________________________________ >>> > > gdal-dev mailing list >>> > > [email protected] >>> > > https://lists.osgeo.org/mailman/listinfo/gdal-dev >>> >>> >>> -- >>> Spatialys - Geospatial professional services >>> http://www.spatialys.com >>> >> > _______________________________________________ > gdal-dev mailing > [email protected]https://lists.osgeo.org/mailman/listinfo/gdal-dev > > _______________________________________________ > gdal-dev mailing list > [email protected] > https://lists.osgeo.org/mailman/listinfo/gdal-dev
_______________________________________________ gdal-dev mailing list [email protected] https://lists.osgeo.org/mailman/listinfo/gdal-dev
