Thanks Daniel and Bjorn,
Will try the increasing the swap file size.
Kind regards,
Kieran Caplice
On 08/03/16 09:51, Daniel Hofmann wrote:
||
|1/ Provide a swapfile (100GB just in case): > fallocate -l 100G
/path/to/swapfile > chmod 600 /path/to/swapfile > mkswap
/path/to/swapfile > swapon /path/to/swapfile |
Note: this does not write 100 GB of zeros. Instead what it does is
allocating a certain amount of blocks and just setting the
'uninitialized' flag on them, returning more or less immediately.
2/ Provide a stxxl config file (250GB should be more than enough, demo
server runs this):
> disk=/path/to/stxxl,250000,syscall
See: http://stxxl.sourceforge.net/tags/master/install_config.html
Here is some additional explanation:
https://github.com/Project-OSRM/osrm-backend/issues/1698#issuecomment-142744361
Cheers,
Daniel J H
On Tue, Mar 8, 2016 at 10:02 AM, Kieran Caplice
<[email protected] <mailto:[email protected]>> wrote:
Hi all,
Just enquiring if anyone has an up-to-date value for the required
RAM to extract/prepare the planet PBF on OSRM v4.9.1? Below is the
output from extraction on a 64 GB RAM machine:
[info] Input file: planet-latest.osm.pbf
[info] Profile: profile.lua
[info] Threads: 12
[info] Using script profile.lua
[STXXL-MSG] STXXL v1.3.1 (release)
[STXXL-MSG] 1 disks are allocated, total space: 250000 MiB
[info] Parsing in progress..
[info] input file generated by planet-dump-ng 1.1.3
[info] timestamp: 2016-02-29T01:59:57Z
[info] Using turn restrictions
[info] Found 3 exceptions to turn restrictions:
[info] motorcar
[info] motor_vehicle
[info] vehicle
[info] Parsing finished after 3630.62 seconds
[info] Raw input contains 3240515916 nodes, 333181812 ways, and
4046532 relations, and 0 unknown entities
[extractor] Sorting used nodes ... ok, after 301.697s
[extractor] Erasing duplicate nodes ... ok, after 262.325s
[extractor] Sorting all nodes ... ok, after 3419.92s
[extractor] Building node id map ... ok, after 1674.42s
[extractor] setting number of nodes ... ok
[extractor] Confirming/Writing used nodes ... ok, after 837.176s
[info] Processed 578533637 nodes
[extractor] Sorting edges by start ... ok, after 1919.47s
[extractor] Setting start coords ... ok, after 2657.74s
[extractor] Sorting edges by target ... ok, after 1894.56s
[extractor] Computing edge weights ... ok, after 2911.29s
[extractor] Sorting edges by renumbered start ... ok, after 1864.9s
[extractor] Writing used edges ... ok, after 557.274s
[extractor] setting number of edges ... ok
[info] Processed 610970822 edges
[extractor] Sorting used ways ... ok, after 89.8886s
[extractor] Sorting 491077 restriction. by from... ok, after 0.906043s
[extractor] Fixing restriction starts ... ok, after 39.6105s
[extractor] Sorting restrictions. by to ... ok, after 0.734124s
[extractor] Fixing restriction ends ... ok, after 40.6053s
[info] usable restrictions: 459264
[extractor] writing street name index ... ok, after 3.3452s
[info] extraction finished after 23181.9s
[info] Generating edge-expanded graph representation
[info] - 459264 restrictions.
[info] Importing n = 578533637 nodes
[info] - 157152 bollard nodes, 792260 traffic lights
[info] and 610970822 edges
[info] Graph loaded ok and has 610970822 edges
[warn] std::bad_alloc
From my reading, this is caused by running out of RAM. The only
files created were:
-rw-r--r-- 1 root root 25G Mar 7 18:57
planet-latest.osrm
-rw-r--r-- 1 root root 118M Mar 7 19:00
planet-latest.osrm.names
-rw-r--r-- 1 root root 15M Mar 7 19:00
planet-latest.osrm.restrictions
-rw-r--r-- 1 root root 20 Mar 7 12:33
planet-latest.osrm.timestamp
Which obviously means preparing gives the following:
[info] Input file: planet-latest.osrm
[info] Profile: profile.lua
[info] Threads: 12
[info] Loading edge-expanded graph representation
[info] Opening planet-latest.osrm.ebg
[warn] [exception] osrm input file misses magic number. Check or
reprocess the file
Thanks.
Kind regards,
Kieran Caplice
_______________________________________________
OSRM-talk mailing list
[email protected] <mailto:[email protected]>
https://lists.openstreetmap.org/listinfo/osrm-talk
_______________________________________________
OSRM-talk mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/osrm-talk
_______________________________________________
OSRM-talk mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/osrm-talk
