1/ Provide a swapfile (100GB just in case): > fallocate -l 100G /path/to/swapfile > chmod 600 /path/to/swapfile > mkswap /path/to/swapfile > swapon /path/to/swapfile
Note: this does not write 100 GB of zeros. Instead what it does is allocating a certain amount of blocks and just setting the 'uninitialized' flag on them, returning more or less immediately. 2/ Provide a stxxl config file (250GB should be more than enough, demo server runs this): > disk=/path/to/stxxl,250000,syscall See: http://stxxl.sourceforge.net/tags/master/install_config.html Here is some additional explanation: https://github.com/Project-OSRM/osrm-backend/issues/1698#issuecomment-142744361 Cheers, Daniel J H On Tue, Mar 8, 2016 at 10:02 AM, Kieran Caplice <[email protected]> wrote: > Hi all, > > Just enquiring if anyone has an up-to-date value for the required RAM to > extract/prepare the planet PBF on OSRM v4.9.1? Below is the output from > extraction on a 64 GB RAM machine: > > [info] Input file: planet-latest.osm.pbf > [info] Profile: profile.lua > [info] Threads: 12 > [info] Using script profile.lua > [STXXL-MSG] STXXL v1.3.1 (release) > [STXXL-MSG] 1 disks are allocated, total space: 250000 MiB > [info] Parsing in progress.. > [info] input file generated by planet-dump-ng 1.1.3 > [info] timestamp: 2016-02-29T01:59:57Z > [info] Using turn restrictions > [info] Found 3 exceptions to turn restrictions: > [info] motorcar > [info] motor_vehicle > [info] vehicle > [info] Parsing finished after 3630.62 seconds > [info] Raw input contains 3240515916 nodes, 333181812 ways, and 4046532 > relations, and 0 unknown entities > [extractor] Sorting used nodes ... ok, after 301.697s > [extractor] Erasing duplicate nodes ... ok, after 262.325s > [extractor] Sorting all nodes ... ok, after 3419.92s > [extractor] Building node id map ... ok, after 1674.42s > [extractor] setting number of nodes ... ok > [extractor] Confirming/Writing used nodes ... ok, after 837.176s > [info] Processed 578533637 nodes > [extractor] Sorting edges by start ... ok, after 1919.47s > [extractor] Setting start coords ... ok, after 2657.74s > [extractor] Sorting edges by target ... ok, after 1894.56s > [extractor] Computing edge weights ... ok, after 2911.29s > [extractor] Sorting edges by renumbered start ... ok, after 1864.9s > [extractor] Writing used edges ... ok, after 557.274s > [extractor] setting number of edges ... ok > [info] Processed 610970822 edges > [extractor] Sorting used ways ... ok, after 89.8886s > [extractor] Sorting 491077 restriction. by from... ok, after 0.906043s > [extractor] Fixing restriction starts ... ok, after 39.6105s > [extractor] Sorting restrictions. by to ... ok, after 0.734124s > [extractor] Fixing restriction ends ... ok, after 40.6053s > [info] usable restrictions: 459264 > [extractor] writing street name index ... ok, after 3.3452s > [info] extraction finished after 23181.9s > [info] Generating edge-expanded graph representation > [info] - 459264 restrictions. > [info] Importing n = 578533637 nodes > [info] - 157152 bollard nodes, 792260 traffic lights > [info] and 610970822 edges > [info] Graph loaded ok and has 610970822 edges > [warn] std::bad_alloc > > From my reading, this is caused by running out of RAM. The only files > created were: > > -rw-r--r-- 1 root root 25G Mar 7 18:57 planet-latest.osrm > -rw-r--r-- 1 root root 118M Mar 7 19:00 > planet-latest.osrm.names > -rw-r--r-- 1 root root 15M Mar 7 19:00 > planet-latest.osrm.restrictions > -rw-r--r-- 1 root root 20 Mar 7 12:33 > planet-latest.osrm.timestamp > > Which obviously means preparing gives the following: > > [info] Input file: planet-latest.osrm > [info] Profile: profile.lua > [info] Threads: 12 > [info] Loading edge-expanded graph representation > [info] Opening planet-latest.osrm.ebg > [warn] [exception] osrm input file misses magic number. Check or reprocess > the file > > Thanks. > > Kind regards, > Kieran Caplice > > > _______________________________________________ > OSRM-talk mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/osrm-talk > >
_______________________________________________ OSRM-talk mailing list [email protected] https://lists.openstreetmap.org/listinfo/osrm-talk
