Even Rouault <even.rouault <at> mines-paris.org> writes:
> > However, the performance on germany.osm.pbf seemed very slow on your PC, but > after testing on mine it takes ~9 hours, which seemed too slow since a > conversion for the full planet-latest.osm.pbf (17 GB) into "null" (this is a > debug output driver, not compiled by default, that doesn't write anything) > has > taken ~ 30h (which, while looking at > http://wiki.openstreetmap.org/wiki/Osm2pgsql/benchmarks, isn't particularly > bad) > > After investigations, most of the slowdown it is due to the building of the > spatial index of the output spatialite DB. When the spatial index is created > at DB initialization, and updated at each feature insertion, the performance > is clearly affected. For example, when adding -lco SPATIAL_INDEX=NO to the > command line, the conversion of germany only takes 2 hours. Adding manually > the spatial index at the end with ogrinfo the.db -sql "SELECT > CreateSpatialIndex('points', 'GEOMETRY')" (and the same for lines, polygons, > multilines, multipolygons, other_relations) takes ~ 22 minutes, so overall, > this is 4 times faster. > > In r24715, I've implemented defered spatial index creation. And indeed the > whole process takes now ~ 2h20. Hi, I borrowed my son's computer and made one more test. Important numbers about the computer: Windows 7, 64 -bit Four-core Intel i5 2500k @3,3 GHz SSD disk Timings for germany.osm.pbf by using -lco SPATIAL_INDEX=NO. Times are total times from the beginning of the test 70% progress - 5 minutes (I suppose resolving ways begins about at this phase) 100 % progress - 17 minutes manual index creation for all the layers ready - 45 minutes Results are interesting. Import to Spatialite without indexing was six times faster for me and I suppose it is mostly because of the SSD drive. But creating indexes took me 30 minutes while you timing was 22 minutes. Perhaps there is something sub-optimal in combination of Windows/Spatialite/Create spatial index. Anyway, the score for germany.osm.pbf is now 45 minutes. Someone with Linux and SSD drive is perhaps the one to beat the record. For comparison, converting finland.osm.pbf took 2 min 40 sec. Converting Germany took 20 times more time and I believe that the relation is OK now. -Jukka Rahkonen- _______________________________________________ gdal-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/gdal-dev
