I did some more testing. I've taken a smaller area and put everything into a tmpfs but even with the .pbf as well as the tmp files of osmosis both being stored in ram, the performance isn't too good. It improved to about 1500 objects/second but this would still means that all ways (according to this [1]) need 113 hours which is way too long.
I've documented the import of a similar file in size (about 4GB) about half a year ago, running on HDD only, which was completed in just over 11 hours. Now this was a different machine but I can't explain, why it is that slow. One thing caught my attention tho, osmosis seems to use only one thread... Not sure, if that's the bottleneck [1] https://taginfo.openstreetmap.org/reports/database_statistics Frederik Ramm: > Hi, > > the osm2city software should be changed to use an osm2pgsql database > instead of an osmosis database. Not only can a planet be imported in > less than a day with osm2pgsql (if you have SSDs), but also the > osm2pgsql database already has correctly built geometries for all > objects, whereas osm2city has to make an effort to build these > geometries from raw OSM data,thereby re-inventing the wheel when it > comes to the interpretation of multipolygon relations, the treatment of > way-based vs. relation-based polygons, etc. > > osm2city does not seem to use anything that could *not* be found in an > osm2pgsql import. > > If you insist on continuing down your current path then you must either > equip your computer with fast SSDs, or temporarily rent a large-SSD > Amazon instance on which you can do your import and then copy over the > resulting database (if you choose a setup where the importing instance > has the same CPU architecture, as well as exactly the same OS and > PostgreSQL/PostGIS versions, then you can copy over the raw database > directory). But even this is likely to take at least a week if not > several for the import - osmosis imports are just not something people > do normally on a planet scale. > > I have only cursorily looked at the osm2city source code and it seems > that it uses most of OSM's data (buildings, roads, landuse). If you > should be in a situation where you only need some of OSM's data then a > speedup could be gained by first running "osmium tags-filter" to extract > the data you really need from the planet file. But if the list of "data > you need" contains roads and buildings and landuse then you might as > well not filter, since those categories make up the bulk of OSM data. > > Bye > Frederik > _______________________________________________ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
