> From: Akos Maroy [mailto:[email protected]] > Subject: [OSM-dev] disk size for planet osm import into PostGIS (on an SSD)? > > Hi, > > I'd like to inquire about the estimated disk size needed to import the planet osm file into PostGIS?
For osm2pgsql about 240GB for the database plus 17GB for flat-nodes is my recollection for the final size from my last tests. > > What I wanted to do is to import planet osm into a PostGIS that is based on a 512GB SSD (well, formatted and reported size: 459G, 480720592 bytes). I did the import using osm2pgsql, using: > > /usr/local/bin/osm2pgsql -d osm_world -s -C 5800 --hstore-all -K -v -G -m planet-130620.osm.bz2 > > the end of the import is the following: > > ... > failed: ERROR: could not extend file "base/1602600/4948340.12": No space left on device > HINT: Check free disk space. > > Error occurred, cleaning up There are a couple of ways to reduce the disk space. The first is to use flat-nodes. This turns what was a 80GB+ nodes table for a full planet into a 17GB flat file. One other optimization is if you're not planning on doing updates and don't need the slim tables you can use --drop to get rid of them. By default osm2pgsql does indexing and clustering of the tables in parallel. This is fastest, but it results in a big spike of disk usage while rearranging and indexing as it happens on all the tables at once. I believe --disable-parallel-indexing will fix this. I am quite surprised you ran into problems on a 512GB SSD. I've imported recent planets on smaller volumes. On the other hand, I don't know anyone who's done a full planet import without --flat-nodes lately and that probably helps lots. Two other tips for the next time you try are to use the .osm.pbf file instead of .osm.bz2, and that there was a new planet file generated about two hours ago. Another general tip is that if your planet file is more than a day or so old, use osmupdate (https://wiki.openstreetmap.org/wiki/Osmupdate) to update your planet file before importing. It only takes about an hour even if it's a week old, and that's way faster than importing diffs after. Because osm2pgsql has *so* many options the help text will now suggest a reasonable command for importing data, see https://github.com/openstreetmap/osm2pgsql/commit/34a30092d6 _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
