I just realized I forgot to include the import command I used. The server I used has 6 cores and 24GB RAM, and I configured 24GB swap as well.
$ osm2pgsql -d planet_osm -C 16000 -s -k -K --cache-strategy dense --number-processes 6 -r pbf planet-121205.osm.pbf I also used the following config tweaks for performance.. shared_buffers = 6GB synchronous_commit = off checkpoint_segments = 64 checkpoint_timeout = 15min checkpoint_completion_target = 0.9 checkpoint_warning = 600s I got plenty of warnings that checkpoints were occurring in the range of 4-5 minutes, so I probably could have bumped up checkpoint_segments. I would really prefer not to spend another couple of days importing, so any ideas on finishing out this import would be greatly appreciated. Shane R. -----Original Message----- From: Rowley, Shane K Sent: Friday, December 21, 2012 11:23 AM To: [email protected] Subject: import crashed at "going over pending ways" I ran an import of the full planet file in slim mode. It ran for 34 hours and seems to have successfully imported the data, but then crashed. Here is the end of the log of the output.. --- BEGIN LOG OUTPUT --- Processing: Node(1686947k 46.2k/s) Way(160474k 4.76k/s) Relation(1696080 31.89/s) parse time: 123407s Node stats: total(1686947380), max(2050735481) in 36526s Way stats: total(160474814), max(194569856) in 33697s Relation stats: total(1696083), max(2619043) in 53184s Committing transaction for planet_osm_point Committing transaction for planet_osm_line Committing transaction for planet_osm_polygon Committing transaction for planet_osm_roads Going over pending ways... --- END LOG OUTPUT --- I am able to see the data in the database, so I know that part was successful. $ psql -d planet_osm psql (9.2.2) Type "help" for help. planet_osm=# select count(*) from planet_osm_point ; count ---------- 40050706 (1 row) planet_osm=# select count(*) from planet_osm_line ; count ---------- 63655784 (1 row) planet_osm=# select count(*) from planet_osm_polygon ; count --------- 1197209 (1 row) planet_osm=# select count(*) from planet_osm_roads ; count --------- 6733858 (1 row) I am wondering if it is possible to resume, or recover, or re-create the remaining steps of the import. I know the very last thing it does is to created the indexes, which I could probably do manually. However, I don't know what it's doing when it's "going over pending ways" and "going over pending relations" (although the pending relations seemed to be an empty set the last time I got this to complete successfully). Any suggestions? Would importing a diff clean this up? Or is that impossible until this is corrected? Thanks for any help.. Shane R. _______________________________________________ Imports mailing list [email protected] http://lists.openstreetmap.org/listinfo/imports
