If osrm-prepare crashed, just add more swap. Also make sure to read this ticket: > https://github.com/Project-OSRM/osrm-backend/issues/1698
I don't think we have documentation for the profiles. I know they are somewhat ugly and huge. I would recommend taking a look at the profile's general structure first, so that you roughly understand e.g. that there are callbacks for OSM nodes and ways, and what they do. After that I would simply trace the globals through the script. That is e.g. search for all places where a certain whitelist is used and build a mental model based on that. Pen and paper may help :) > https://github.com/Project-OSRM/osrm-backend/blob/develop/profiles/car.lua You can find the C++ side of things in the extractor directory: > https://github.com/Project-OSRM/osrm-backend/tree/develop/extractor For example, the scripting environment uses luabind to make a few functions visible, e.g. the get_value_by_key function, that then can be used in the profiles. The other way, that is using lua functions in C++ is mostly done in the extractor implementation: I would recommend just searching for the symbols you want to know more about: > https://github.com/Project-OSRM/osrm-backend/search?utf8=%E2%9C%93&q=node_function&type=Code Cheers, Daniel On Mon, Oct 12, 2015 at 4:00 AM, Richard Marsden <[email protected]> wrote: > It is possible I've just managed to demonstrate this to myself. > osrm-prepare crashed out on the Europe-wide dataset at the 90+% > complete level with 30GB swap (16GB real). For much of the run it > didn't go above about 15GB swap. Very slow of course - but that was to > be expected. > > This continues to be a side / evaluation project, but I'm continuing > to investigate. Looks like the PC in question should be able to handle > most countries, but will balk at most continents, and the very largest > countries (e.g. the US). High-RAM hardware is relatively pricey, but > it is a possibility in the new year. > > Looking at the individual pieces, next I'm going to look at the > profile.lua file. I've never used LUA before, but that is easy enough > to remedy with online tutorials and the various books that are > available. But what about the script that is used by OSRM? Is there > any documentation as to the variables and/or functions that are > expected? And the tags used in OSM that the script is expected to > map? > > > Cheers, > > Richard Marsden > > On Thu, Sep 17, 2015 at 4:37 PM, Patrick Niklaus > <[email protected]> wrote: > > W.r.t. the pre-preprocessing you are correct. > > > >> What is that extra power used for? > > > > Including all sorts of external data sources. Also the logic in the > > lua profiles is not just replaceable by simple key-value pairs, OSM > > requires you to handle a lot of special cases. > > > >> Presumably I could do the same for world preparation & routing? Have, > perhaps a 100GB+ swap file, ideally on an SSD. > > > > This will fall apart when you have some actual load pressure on the > > system. We need random access to memory, which will create a lot of > > page faults (== slow). Even an SSD is not even close to memory speed. > > > > You have two options: > > - split the datasets > > - get a bigger server > > > > Cheers, > > Patrick > > > > > > On Thu, Sep 17, 2015 at 10:06 PM, Richard Marsden <[email protected]> > wrote: > >> I've been evaluating OSRM, using it primarily as a library from C++. > >> > >> I believe I've determined the answer to most of the questions, but I'm > >> also looking for confirmation. > >> (I understand the reason for these constraints - the trade-off of > >> speed vs flexibility) > >> > >> First, road speeds are set with 'profile.lua' at the osrm-extract > >> stage. This filters out unnecessary roads (eg. foot paths for car > >> routing), but also applies the road speeds. > >> If I wish to change the speed profile, I need to regenerate the road > >> network with osrm-extract and osrm-routed. > >> Correct? > >> > >> If I wanted different speeds for the final distance/time calculations, > >> I could use the returned route, and apply my own speed table according > >> to the road type of each road segment. This would not, of course, > >> change the route geometry is calculated. > >> > >> If I want a shortest route (distance optimized) instead of a quickest > >> route (time optimized), I need to set all the road speeds to the same > >> speed and regenerate the network. I.e. osrm does not directly support > >> the concept of a "shortest route". > >> > >> The profile is provided with a LUA file. I had to look this one up :-) > >> Looks a useful scripting language, but why is this profile a script > >> file, and not a simple configuration file of constants (eg. key-value > >> pairs)? > >> Seems like an unnecessary complexity - I'd like to understand the > >> perceived advantages. What is that extra power used for? > >> > >> Finally, the memory usage... I saw a reference to the server requiring > >> 40GB of memory for pan-European routing. Presumably that could be > >> offset with a large swap file(?) > >> A large swap file has worked well when I was testing the US-South > >> region on an 8GB machine. > >> Presumably I could do the same for world preparation & routing? Have, > >> perhaps a 100GB+ swap file, ideally on an SSD. > >> > >> > >> Cheers, > >> > >> Richard Marsden > >> > >> _______________________________________________ > >> OSRM-talk mailing list > >> [email protected] > >> https://lists.openstreetmap.org/listinfo/osrm-talk > > > > _______________________________________________ > > OSRM-talk mailing list > > [email protected] > > https://lists.openstreetmap.org/listinfo/osrm-talk > > _______________________________________________ > OSRM-talk mailing list > [email protected] > https://lists.openstreetmap.org/listinfo/osrm-talk >
_______________________________________________ OSRM-talk mailing list [email protected] https://lists.openstreetmap.org/listinfo/osrm-talk
