Hi again, Patrick, On Fri, May 11, 2018 at 9:22 PM, Patrick Agin <[email protected]> wrote: > > And are you aware of some python code that would do the calls to osrm-routed > in parallel threads?
There are lot of examples of how to make python things in parallel threads. For example: https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python Sasha > Regards, > Patrick > > 2018-05-11 14:07 GMT-04:00 Daniel Patterson <[email protected]>: >> >> Patrick, >> >> There are about a million possible paths you could take here, a lot of >> it will depend on what skills you have available. Off the top of my head: >> >> 1) Speed things up by avoiding HTTP overhead and calling the libosrm.a >> functions directly instead of hitting `osrm-routed` over HTTP >> 2) Modify the OSRM C++ source code and strip out the parts of the >> map-matching response you don't need >> 3) Simplify your trace geometries to speed up map-matching >> 4) Break your trace list into sets and run these on multiple machines >> in parallel (make copies of the OSRM data onto multiple machines) >> 5) Just wait 10 hours, and get a good nights sleep >> >> libosrm.a is thread-safe, so if you're calling functions from threads >> you can do many at once. >> >> osrm-routed is multi-threaded, so you can run many queries in parallel - >> how many will depend on how many CPUs your machine has. Profiling >> multi-threaded server performance is kind of beyond the scope of OSRM >> itself, there is lots of literature on it. >> >> daniel >> >> On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <[email protected]> >> wrote: >>> >>> Sorry for the newbie question but what's the difference between >>> osrm-routed and libosrm? Is it mandatory to use the latter for a parallel >>> usage? And do you have an example of code that does the calls in parallel >>> threads? Thanks Sasha for your help. >>> Patrick >>> >>> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <[email protected]>: >>>> >>>> Hi Patrick, >>>> >>>> If you are using libosrm (which reported to be thread safe: >>>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can just >>>> split your list and run its parts in different parallel threads. >>>> >>>> Sasha >>>> >>>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <[email protected]> >>>> wrote: >>>> > Hi, >>>> > I want to call the map matching service for nearly a million of >>>> > routes. I >>>> > calculated that the task could take 10 hours on my laptop to complete. >>>> > I was >>>> > wondering if there's someone who faced the same problem and who >>>> > addressed it >>>> > in parallel, maybe with Spark? Another tool or way? >>>> > Any hint is greatly appreciated! >>>> > Patrick >>>> > >>>> > _______________________________________________ >>>> > OSRM-talk mailing list >>>> > [email protected] >>>> > https://lists.openstreetmap.org/listinfo/osrm-talk >>>> > >>>> >>>> _______________________________________________ >>>> OSRM-talk mailing list >>>> [email protected] >>>> https://lists.openstreetmap.org/listinfo/osrm-talk >>> >>> >>> >>> _______________________________________________ >>> OSRM-talk mailing list >>> [email protected] >>> https://lists.openstreetmap.org/listinfo/osrm-talk >>> >> >> >> _______________________________________________ >> OSRM-talk mailing list >> [email protected] >> https://lists.openstreetmap.org/listinfo/osrm-talk >> > > > _______________________________________________ > OSRM-talk mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/osrm-talk > _______________________________________________ OSRM-talk mailing list [email protected] https://lists.openstreetmap.org/listinfo/osrm-talk
