I did some more testing.
I've taken a smaller area and put everything into a tmpfs but even with
the .pbf as well as the tmp files of osmosis both being stored in ram,
the performance isn't too good. It improved to about 1500 objects/second
but this would still means that all ways (according to this [1]) need
113 hours which is way too long.

I've documented the import of a similar file in size (about 4GB) about
half a year ago, running on HDD only, which was completed in just over
11 hours.

Now this was a different machine but I can't explain, why it is that
slow. One thing caught my attention tho, osmosis seems to use only one
thread... Not sure, if that's the bottleneck

[1] https://taginfo.openstreetmap.org/reports/database_statistics

Frederik Ramm:
> Hi,
> the osm2city software should be changed to use an osm2pgsql database
> instead of an osmosis database. Not only can a planet be imported in
> less than a day with osm2pgsql (if you have SSDs), but also the
> osm2pgsql database already has correctly built geometries for all
> objects, whereas osm2city has to make an effort to build these
> geometries from raw OSM data,thereby re-inventing the wheel when it
> comes to the interpretation of multipolygon relations, the treatment of
> way-based vs. relation-based polygons, etc.
> osm2city does not seem to use anything that could *not* be found in an
> osm2pgsql import.
> If you insist on continuing down your current path then you must either
> equip your computer with fast SSDs, or temporarily rent a large-SSD
> Amazon instance on which you can do your import and then copy over the
> resulting database (if you choose a setup where the importing instance
> has the same CPU architecture, as well as exactly the same OS and
> PostgreSQL/PostGIS versions, then you can copy over the raw database
> directory). But even this is likely to take at least a week if not
> several for the import - osmosis imports are just not something people
> do normally on a planet scale.
> I have only cursorily looked at the osm2city source code and it seems
> that it uses most of OSM's data (buildings, roads, landuse). If you
> should be in a situation where you only need some of OSM's data then a
> speedup could be gained by first running "osmium tags-filter" to extract
> the data you really need from the planet file. But if the list of "data
> you need" contains roads and buildings and landuse then you might as
> well not filter, since those categories make up the bulk of OSM data.
> Bye
> Frederik

dev mailing list

Reply via email to