OSRM pre-processing is kind of worst-case for disk swapping - it does a lot of random access, so if you run out of RAM, pages will get continuously swapped in/out. The slowdown between RAM-only and swap-on-SSD can be 50x or more. The difference between swap on a spinning disk and an SSD is a good 10x on top of that.
If you can, avoid letting OSRM pre-processing touch swap at all costs - it's worth renting a big machine on AWS/Google/Azure for a short time to pre-process the data. daniel On Fri, Sep 22, 2017 at 7:14 AM, Sayer, Bryan <[email protected]> wrote: > Just as a point of reference, you mention that avoiding swapping and just > using RAM will be much faster. If the swap space is an SSD drive, how much > does using only RAM speed up contract? That is, what is the difference > between using say a 7200 rpm SATA drive versus an SSD drive make? > ------------------------------ > *From:* Daniel Patterson <[email protected]> > *Sent:* Friday, September 22, 2017 8:05:56 AM > *To:* Mailing list to discuss Project OSRM > *Subject:* Re: [OSRM-talk] Expecting time for osrm-contract for planet > > Hi Kieran, > > We don't, but I've done some profiling in the past, Docker itself should > have negligible impact. > > The two things that might be slowing things down: > > 1) The docker images don't bundle jemalloc (http://jemalloc.net/), > which can speed things up by 10-15% > 2) osrm-contract is very CPU intensive - the more threads you give it, > the faster it will run. Give it more if you can. > > 256GB of RAM should be more than enough to avoid swapping, but you might > want to disable swap to be sure - if it's being used, it will have a huge > impact on speed. > > daniel > > On Fri, Sep 22, 2017 at 2:30 AM, Kieran Caplice < > [email protected]> wrote: > >> Hi Daniel, >> Can you clarify if you use Docker at Mapbox? What kind of server do you >> guys have (if not the one described on the wiki page)? >> >> Over night, the process has gone to just 75% complete from 65% yesterday >> - 41 hours in total now, and back to maxing CPU again. >> >> Kind regards, >> Kieran Caplice >> >> On 21/09/17 17:17, Daniel Patterson wrote: >> >> OSRM supports *two* core routing algorithms - CH and MLD. The >> `osrm-contract` tool generates the CH dataset, but you can use the MLD >> pipeline instead with: >> >> osrm-extract -p profiles/bicycle.lua yourmap.osm.pbf >> osrm-partition yourmap.osrm >> osrm-customize yourmap.osrm >> osrm-routed -a MLD yourmap.osrm >> >> This sequence of tools should be significantly quicker than osrm-contract >> - the price you pay is that routing requests are about 5x slower (still >> pretty fast though!). The reason that MLD exists is for the >> `osrm-customize` step - it allows you to import traffic data very quickly >> and update the routing graph (~1 minute for North America). >> >> It's hard to say exactly what's going wrong with osrm-contract here - >> here at Mapbox, we daily run `osrm-contract` over the latest planet with >> the bicycle profile without a problem, however, Alex and others have >> reported issues with what seem like hangs on much smaller datasets that >> we've been unable to reproduce so far. >> >> The runtime of osrm-contract is affected by how much hierarchy exists in >> the data - the more similar the edge speeds (like in foot) and the more >> edges there are, the slower it gets, often in a non-linear fashion. The >> car profile has a very hierarchical structure (many different road speeds), >> so it fits well into the CH, the construction algorithm doesn't need to >> compare as many options. >> >> daniel >> >> On Thu, Sep 21, 2017 at 9:03 AM, Kieran Caplice < >> [email protected]> wrote: >> >>> We're actually looking for the best of both car and foot, so in my head, >>> bicycle would be the happy medium (though I could be completely wrong on >>> this). >>> Kind regards, >>> Kieran Caplice >>> >>> On 21/09/17 16:53, Alex Farioletti wrote: >>> >>> i've run into the same issues, and now i just use metroextracts of the >>> areas that i need for the bike stuff i do and it reduces the time >>> significantly >>> >>> *Alex Farioletti* >>> *415.312.1674* >>> *tcbcourier.com <http://tcbcourier.com> * >>> >>> On Thu, Sep 21, 2017 at 8:49 AM, Kieran Caplice < >>> [email protected]> wrote: >>> >>>> Thanks Daniel. >>>> >>>> I'm using the bicycle profile, so I would expect based on what you've >>>> said that somewhere up to 36 hours would be likely. However, this is the >>>> current output, after 25h40m: >>>> [info] Input file: /data/1505492056/planet-latest.osrm >>>> [info] Threads: 12 >>>> [info] Reading node weights. >>>> [info] Done reading node weights. >>>> [info] Loading edge-expanded graph representation >>>> [info] merged 2379332 edges out of 1777752432 >>>> [info] initializing node priorities... ok. >>>> [info] preprocessing 389797971 (90%) nodes... >>>> [info] . 10% . 20% . 30% . 40% . 50% . 60% >>>> >>>> It hasn't advanced past 60% in the last 2-3 hours. It is however maxing >>>> CPU and using approximately the same amount of RAM since it started. >>>> >>>> Kind regards, >>>> Kieran Caplice >>>> >>>> On 21/09/17 16:39, Daniel Patterson wrote: >>>> >>>> Hi Kieran, >>>> >>>> The contraction time will be slow - many, many hours for the whole >>>> planet. *Typically* for the car profile it's about 12 hours, but if you >>>> use bike or foot, or your own profile, it can get a lot bigger. >>>> >>>> If you've messed with the travel speeds, that can have a big effect >>>> too. 24 hours is not unheard of, but whether it's legit will depend a lot >>>> on the details. >>>> >>>> daniel >>>> >>>> On Thu, Sep 21, 2017 at 7:00 AM, Kieran Caplice < >>>> [email protected]> wrote: >>>> >>>>> Hi all, >>>>> >>>>> Could anyone give an approx estimate for the time required to run the >>>>> osrm-contract on planet data on a 12 thread, 256 GB RAM, SSD machine? >>>>> >>>>> The osrm-extract process finished in 232 minutes, but the contract has >>>>> now been running solid for 24 hours, and appears to be stuck at 60% on >>>>> "preprocessing nodes". All 12 cores are generally maxed out, and the >>>>> process is using nearly 90 GB of RAM. >>>>> >>>>> This is the second time I've run the contract process, as my SSH >>>>> connection to the server dropped the first time and the process wasn't >>>>> running in a screen etc, so I assumed after the 40-odd hours it was >>>>> running >>>>> for, the connection drop caused it to hang, but now I'm not so sure. Were >>>>> there any files I should maybe have cleared before trying to run it again? >>>>> >>>>> I'm using the docker image to run the command (using >>>>> osrm/osrm-backend:latest): time docker run -t -v /opt/osrm/data:/data >>>>> osrm/osrm-backend osrm-contract /data/1505492056/planet-latest.osrm >>>>> >>>>> Kind regards, >>>>> Kieran Caplice >>>>> >>>>> >>>>> _______________________________________________ >>>>> OSRM-talk mailing list >>>>> [email protected] >>>>> https://lists.openstreetmap.org/listinfo/osrm-talk >>>>> >>>> >>>> >>>> >>>> _______________________________________________ >>>> OSRM-talk mailing >>>> [email protected]https://lists.openstreetmap.org/listinfo/osrm-talk >>>> >>>> >>>> >>>> _______________________________________________ >>>> OSRM-talk mailing list >>>> [email protected] >>>> https://lists.openstreetmap.org/listinfo/osrm-talk >>>> >>>> >>> >>> >>> _______________________________________________ >>> OSRM-talk mailing >>> [email protected]https://lists.openstreetmap.org/listinfo/osrm-talk >>> >>> >>> >>> _______________________________________________ >>> OSRM-talk mailing list >>> [email protected] >>> https://lists.openstreetmap.org/listinfo/osrm-talk >>> >>> >> >> >> _______________________________________________ >> OSRM-talk mailing >> [email protected]https://lists.openstreetmap.org/listinfo/osrm-talk >> >> >> >> _______________________________________________ >> OSRM-talk mailing list >> [email protected] >> https://lists.openstreetmap.org/listinfo/osrm-talk >> >> > > ****WARNING**** This information may be confidential. It is intended only > for the addressee(s) identified above. If you are not the addressee(s), or > an employee or agent of the addressee(s), please note that any > dissemination, distribution, or copying of this communication is strictly > prohibited. If you have received this information in error, please destroy > the information and notify the sender of the error. Thank you. > > _______________________________________________ > OSRM-talk mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/osrm-talk > >
_______________________________________________ OSRM-talk mailing list [email protected] https://lists.openstreetmap.org/listinfo/osrm-talk
