On Fri, Feb 6, 2009 at 5:20 PM, Marcus Wolschon <mar...@wolschon.biz> wrote: > On Fri, Feb 6, 2009 at 4:35 PM, Tom Hughes <t...@compton.nu> wrote: >> Erik Johansson wrote: >> >>> How much of the DB load comes from the read only part of the API, and >>> what if you remove the area limit on the map call? >> >> If I remove the area limit then somebody will do a massive query that >> will suck up all the memory on the machine and everything will die. >> >> That limit is not (primarily) about the cost of gathering the required >> data, though obviously that might become an issue as well, it's about >> the fact that we are holding the whole result set in memory (several >> times over in fact) on the rails server and those servers only have a >> finite amount of memory. > > I'm not sure I understand why you do this. > The most expensive call would probably be /map with a bounding-box. > You query all the node, stream them out and keep the nodeIDs > (but not the whole resultset with lat, lon, version, ...) in memory. > Query the ways that use these nodes, keeping their IDs in memory. > Then Query the relations, stream them out and forget about the nodes > and ways while doing that.
in an ideal world. however, there are a few issues; Frederik has already raised the point about sorting by ID, Rails doesn't (yet?) support streaming results and you cannot indicate an error condition (e.g: MySQL dies, bad UTF-8) in the response header once you've already sent part of the body. > Anyway, the question was on how much of the current load is done by > non-writing queries (not just /map). Do we have an answer for that? not directly, but by comparing http://munin.openstreetmap.org/openstreetmap/db.openstreetmap-load.html http://munin.openstreetmap.org/openstreetmap/db.openstreetmap-mysql_queries.html we can see that about 75% of all queries are selects, but that times of high load appear better correlated with inserts and deletes. it is difficult to quantify how much load is purely due to reads, as part of the load may be due to contention between the reads and writes. cheers, matt _______________________________________________ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
