On Fri, May 6, 2011 at 2:48 PM, James Ewen <[email protected]> wrote: > Is there a way to know how much load is being exerted by each type of > application out there?
The sysadmins know, but I expect that the logs can't be exposed due to privacy law. > We have a newer application that uses OSM maps. The author is aware of > the usage policy, and is trying to be gentle on the servers, but as > the number of people running the application grows, the demand on the > server grows. > > It would be good to be able to see which applications are putting > unreasonable burdens on the server. We've been reaching out to the worst offenders and getting them to correct their behaviour. Sadly, apps are popping up faster than we can sort them out. All users are now rate limited, but the apps make demands that are so out of perspective that even rate limits don't eliminate the harm they do. Here's an example. Of our max-zoom tiles z18, only a few percent are ever viewed. Ever. By anybody. So we don't render them except on demand. That's expensive to do in terms of server resources, but mappers love it. Mappers can see their mapping results right away and that means we get better data, faster. Bulk downloaders take 100% of the tiles in their area of interest. We know only a few percent will actually be viewed. But they ask us to render them all. That takes tiles away from mappers. More specifically, that's why Bernie is still waiting for his tiles today. Have a look at the render queue. http://munin.openstreetmap.org/openstreetmap/yevaud.openstreetmap/renderd_queue.html It's been running flat out for weeks. When we added resources last year, it helped for a bit, but more apps keep popping up and taking our resources. We cannot win that race. > The program by default downloads and caches tiles locally, but purges > the tiles after a day or two. We can over ride the purge, which I do, > keeping tiles for ever. The program can query for tile updates, which > means changes get propagated even when the tiles are stored for ever. Be sure to join #osm-dev on irc.oftc.net and ask for help from the sys admins. They'll get you sorted out. Any application that bulk downloads tiles from the OSM servers is currently at risk of being cut off without further warning. The answer is longer and more complicated than this, but this is what it boils down to. _______________________________________________ Talk-ca mailing list [email protected] http://lists.openstreetmap.org/listinfo/talk-ca
