Hi Kieran, I added a high speed SSD and pointed the .stxxl towards that to deal with the swap slowdown. That bought me a drop in processing time from 12 hrs -> 3-4 hrs.
osrm@mat4:~/osrm-backend$ cat .stxxl *disk=/mnt/tmp/stxxl,400000,syscall* osrm@mat4:~/osrm-backend$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 223.6G 0 disk ├─sda1 8:1 0 4G 0 part │ └─md0 9:0 0 4G 0 raid1 [SWAP] ├─sda2 8:2 0 512M 0 part │ └─md1 9:1 0 511.7M 0 raid1 /boot └─sda3 8:3 0 219.1G 0 part └─md2 9:2 0 219G 0 raid1 / sdb 8:16 0 223.6G 0 disk ├─sdb1 8:17 0 4G 0 part │ └─md0 9:0 0 4G 0 raid1 [SWAP] ├─sdb2 8:18 0 512M 0 part │ └─md1 9:1 0 511.7M 0 raid1 /boot └─sdb3 8:19 0 219.1G 0 part └─md2 9:2 0 219G 0 raid1 / *sdc 8:32 0 447.1G 0 disk * *└─sdc1 8:33 0 447.1G 0 part /mnt* On Wed, Mar 2, 2016 at 4:51 PM, Kieran Caplice <[email protected]> wrote: > Hello, > > I'm currently extracting the planet PBF (~31 GB), and it's been running > for hours. I notice in the "Running OSRM" wiki page, it says " On a Core i7 > with 8GB RAM and (slow) 5400 RPM Samsung SATA hard disks it took about 65 > minutes to do so from a PBF formatted planet", which is making me wonder > why it's taking so long on our server. Below are some example output > messages: > > [info] Parsing finished after 3584.35 seconds > [extractor] Erasing duplicate nodes ... ok, after 319.091s > [extractor] Sorting all nodes ... ok, after 3632.87s > [extractor] Building node id map ... ok, after 2025.29s > [extractor] Confirming/Writing used nodes ... ok, after 1096.24s > [extractor] Sorting edges by start ... ok, after 2000.08s > > Some stxxl errors were outputted as I set the disk size to 100GB thinking > it was enough - but I didn't think it would cause such slowdowns as this, > considering extracting the Europe PBF takes hours also without the stxxl > errors. > > Server specs: > Ubuntu 14.04 > Intel Xeon CPU E5-1650 v3 @ 3.50GHz (hex-core with HT) > 64 GB RAM @ 2133 MHz > 2 TB Western Digital Enterprise 7200 RPM hard drive > > At the moment, disk IO is averaging around 35-40 MB/s R/W (~90%). > > Anyone have any ideas as to what might be going on? Or is it normal to > take this long without an SSD? > > Thanks in advance. > > Kind regards, > Kieran Caplice > > > _______________________________________________ > OSRM-talk mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/osrm-talk > > -- Bjorn Madsen Director, Global Operations Multi-Agent Technology Ltd. Bessemer Drive, Stevenage, SG1 2DX [email protected] Mobile: +44 779 20 307 20 Main Office: +44 1438 310 035
_______________________________________________ OSRM-talk mailing list [email protected] https://lists.openstreetmap.org/listinfo/osrm-talk
