Thank you Frederik. It was about "out of memory". I though have a 4GB swap partition. I'll try again with smaller memory reservations. And actually without coastline and hstore. I also added the fsync=off.
How could I monitor the import process and memory usage so that I could optimize the configurations? Thanks. I'll report if this works or not. On Wed, Jun 6, 2012 at 11:34 AM, Frederik Ramm <[email protected]> wrote: > Hi, > > > On 06/06/2012 07:46 AM, Lauri Kajan wrote: >> >> I have some trouble to import osm data to my postgis database. >> I am trying to import europe dump from gefabrik but the process was >> killed. >> Is this a memory issue or something else? > > > Likely a memory issue; type "dmesg" and you will probably see a message > about an out-of-memory condition that made Linux kill your process. > > >> Is there any possibilities to continue the process from the point of >> the crash? Or should I start all over again? > > > I'm afraid you will have to start over. > > >> I used latest revision 28427 of the osm2pgsql. >> I have 8GB of RAM on my server and I used these configurations on my >> postgresql: >> shared_buffers = 1024MB >> work_mem = 8MB >> maintenance_work_mem = 1024MB >> checkpoint_segments = 20 >> autovacuum = off >> Are these settings a way off? > > > They are ok but if you have Postgres 9.0 or higher, my experience is that > smaller shared_buffers values work just as well. Also add "fsync=off". > > >> And this was the command: >> osm2pgsql --create --database osm --username osm --slim --cache 4096 >> --hstore --number-processes 2 -l --keep-coastlines europe.osm.pbf > > > You will probably have to reduce the cache parameter to avoid the out of > memory situation. (Having a little swap space configured on your system will > also help - if you have swap space then exhausting physical memory will make > your system slow down but not kill a process.) > > You could propbably speed up the process by > > * not keeping coastlines > * modifying your default.style file to delete unwanted tags (with --hstore, > a lot of rubbish like "xyz-import-uuid=some-long-hex-string" is imported > into your database) > * cut out a smaller area from the Europe file (using Osmosis) and import > that, instead of all Europe > * storing the database on an SSD, or if you don't have SSD but you have more > than one disk, then on a Linux software RAID0 built from partitions on two > or more disks > > Bye > Frederik > > PS: A list of frequent but useless tags for adding to default.style, as > generated by Sven Geggus. Works only with newer osm2pgsql versions that have > wildcard support, and only makes sense when you use --hstore: > > node,way 3dshapes:* text delete > node,way AND_nosr_r text delete > node,way b5m:* text delete > node,way bak:* text delete > node,way canvec:* text delete > node,way created_by text delete > node,way dcgis:* text delete > node,way FIXME text delete > node,way geobase:* text delete > node,way gnis:* text delete > node,way kms:* text delete > node,way KSJ2:* text delete > node,way massgis:* text delete > node,way mvdgis:* text delete > node,way naptan:* text delete > node,way ngbe:* text delete > node,way nhd:* text delete > node,way NHD:* text delete > node,way note text delete > node,way note:* text delete > node,way notes text delete > node,way osak:* text delete > node,way source text delete > node,way source:* text delete > node,way sourcedb:* text delete > node,way tiger:* text delete > node,way WroclawGIS:* text delete > node,way yh:* text delete > node,way rednap:* text delete > node,way ideewfs:* text delete > node,way ign:* text delete > > -- > Frederik Ramm ## eMail [email protected] ## N49°00'09" E008°23'33" > > _______________________________________________ > dev mailing list > [email protected] > http://lists.openstreetmap.org/listinfo/dev _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
