thanks Sasha. And do you know if it's possible to do calls to libosrm functions directly from Python?
2018-05-11 14:33 GMT-04:00 Sasha Khapyorsky <[email protected]>: > Hi again, Patrick, > > On Fri, May 11, 2018 at 9:22 PM, Patrick Agin <[email protected]> > wrote: > > > > And are you aware of some python code that would do the calls to > osrm-routed > > in parallel threads? > > There are lot of examples of how to make python things in parallel > threads. For example: > https://stackoverflow.com/questions/2846653/how-to-use-threading-in-python > > Sasha > > > Regards, > > Patrick > > > > 2018-05-11 14:07 GMT-04:00 Daniel Patterson <[email protected]>: > >> > >> Patrick, > >> > >> There are about a million possible paths you could take here, a lot of > >> it will depend on what skills you have available. Off the top of my > head: > >> > >> 1) Speed things up by avoiding HTTP overhead and calling the > libosrm.a > >> functions directly instead of hitting `osrm-routed` over HTTP > >> 2) Modify the OSRM C++ source code and strip out the parts of the > >> map-matching response you don't need > >> 3) Simplify your trace geometries to speed up map-matching > >> 4) Break your trace list into sets and run these on multiple > machines > >> in parallel (make copies of the OSRM data onto multiple machines) > >> 5) Just wait 10 hours, and get a good nights sleep > >> > >> libosrm.a is thread-safe, so if you're calling functions from threads > >> you can do many at once. > >> > >> osrm-routed is multi-threaded, so you can run many queries in > parallel - > >> how many will depend on how many CPUs your machine has. Profiling > >> multi-threaded server performance is kind of beyond the scope of OSRM > >> itself, there is lots of literature on it. > >> > >> daniel > >> > >> On Fri, May 11, 2018 at 10:57 AM, Patrick Agin <[email protected]> > >> wrote: > >>> > >>> Sorry for the newbie question but what's the difference between > >>> osrm-routed and libosrm? Is it mandatory to use the latter for a > parallel > >>> usage? And do you have an example of code that does the calls in > parallel > >>> threads? Thanks Sasha for your help. > >>> Patrick > >>> > >>> 2018-05-11 13:50 GMT-04:00 Sasha Khapyorsky <[email protected]>: > >>>> > >>>> Hi Patrick, > >>>> > >>>> If you are using libosrm (which reported to be thread safe: > >>>> https://github.com/Project-OSRM/osrm-backend/issues/4966) you can > just > >>>> split your list and run its parts in different parallel threads. > >>>> > >>>> Sasha > >>>> > >>>> On Fri, May 11, 2018 at 8:14 PM, Patrick Agin <[email protected] > > > >>>> wrote: > >>>> > Hi, > >>>> > I want to call the map matching service for nearly a million of > >>>> > routes. I > >>>> > calculated that the task could take 10 hours on my laptop to > complete. > >>>> > I was > >>>> > wondering if there's someone who faced the same problem and who > >>>> > addressed it > >>>> > in parallel, maybe with Spark? Another tool or way? > >>>> > Any hint is greatly appreciated! > >>>> > Patrick > >>>> > > >>>> > _______________________________________________ > >>>> > OSRM-talk mailing list > >>>> > [email protected] > >>>> > https://lists.openstreetmap.org/listinfo/osrm-talk > >>>> > > >>>> > >>>> _______________________________________________ > >>>> OSRM-talk mailing list > >>>> [email protected] > >>>> https://lists.openstreetmap.org/listinfo/osrm-talk > >>> > >>> > >>> > >>> _______________________________________________ > >>> OSRM-talk mailing list > >>> [email protected] > >>> https://lists.openstreetmap.org/listinfo/osrm-talk > >>> > >> > >> > >> _______________________________________________ > >> OSRM-talk mailing list > >> [email protected] > >> https://lists.openstreetmap.org/listinfo/osrm-talk > >> > > > > > > _______________________________________________ > > OSRM-talk mailing list > > [email protected] > > https://lists.openstreetmap.org/listinfo/osrm-talk > > > > _______________________________________________ > OSRM-talk mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/osrm-talk >
_______________________________________________ OSRM-talk mailing list [email protected] https://lists.openstreetmap.org/listinfo/osrm-talk
