Thanks Daniel. My first idea was to break my list into sets and running the job on different machines as you said in 4). But out of curiosity, can I make calls to libosrm.a functions directly from Python? And are you aware of some python code that would do the calls to osrm-routed in parallel threads? Regards, Patrick
2018-05-11 14:07 GMT-04:00 Daniel Patterson <dan...@mapbox.com>: > Patrick, > > There are about a million possible paths you could take here, a lot of > it will depend on what skills you have available. Off the top of my head: > > 1) Speed things up by avoiding HTTP overhead and calling the libosrm.a > functions directly instead of hitting `osrm-routed` over HTTP > 2) Modify the OSRM C++ source code and strip out the parts of the > map-matching response you don't need > 3) Simplify your trace geometries to speed up map-matching > 4) Break your trace list into sets and run these on multiple machines > in parallel (make copies of the OSRM data onto multiple machines) > 5) Just wait 10 hours, and get a good nights sleep > > libosrm.a is thread-safe, so if you're calling functions from threads > you can do many at once. > > osrm-routed is multi-threaded, so you can run many queries in parallel - > how many will depend on how many CPUs your machine has. Profiling > multi-threaded server performance is kind of beyond the scope of OSRM > itself, there is lots of literature on it. > > daniel > > On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <agin.patr...@gmail.com> > wrote: > >> Sorry for the newbie question but what's the difference between >> osrm-routed and libosrm? Is it mandatory to use the latter for a parallel >> usage? And do you have an example of code that does the calls in parallel >> threads? Thanks Sasha for your help. >> Patrick >> >> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <sash...@gmail.com>: >> >>> Hi Patrick, >>> >>> If you are using libosrm (which reported to be thread safe: >>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can just >>> split your list and run its parts in different parallel threads. >>> >>> Sasha >>> >>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <agin.patr...@gmail.com> >>> wrote: >>> > Hi, >>> > I want to call the map matching service for nearly a million of >>> routes. I >>> > calculated that the task could take 10 hours on my laptop to complete. >>> I was >>> > wondering if there's someone who faced the same problem and who >>> addressed it >>> > in parallel, maybe with Spark? Another tool or way? >>> > Any hint is greatly appreciated! >>> > Patrick >>> > >>> > _______________________________________________ >>> > OSRM-talk mailing list >>> > OSRM-talk@openstreetmap.org >>> > https://lists.openstreetmap.org/listinfo/osrm-talk >>> > >>> >>> _______________________________________________ >>> OSRM-talk mailing list >>> OSRM-talk@openstreetmap.org >>> https://lists.openstreetmap.org/listinfo/osrm-talk >>> >> >> >> _______________________________________________ >> OSRM-talk mailing list >> OSRM-talk@openstreetmap.org >> https://lists.openstreetmap.org/listinfo/osrm-talk >> >> > > _______________________________________________ > OSRM-talk mailing list > OSRM-talk@openstreetmap.org > https://lists.openstreetmap.org/listinfo/osrm-talk > >
_______________________________________________ OSRM-talk mailing list OSRM-talk@openstreetmap.org https://lists.openstreetmap.org/listinfo/osrm-talk
