Hi NopMap On 27 April 2011 09:29, NopMap <[email protected]> wrote: > > Hi! > > Paging the admins of the OSM tile servers. I'd like to get in touch about > the topic of mass downloaders. It appears that some problems first become > visible on small tile servers like my own, but hit the main tile servers a > while later, so it might be a good thing to share experiences. E.g. MOABC > was an issue for my server in december and had to be blocked on the main > servers about three months later. >
Yes, the MOBAC app is becoming a serious problem for the tile.osm.org server. Earlier versions fake a Firefox user-agent. I have had to reduce max tile QOS limit downward in an attempt to control the situation. Allowing mass tile scraping is not sustainable. A lively thread when I restricted the Faked user-agent: http://sourceforge.net/projects/mobac/forums/forum/861096/topic/4423122 tile.osm.org traffic has recently peaked around 300Mb/s. > A few days ago a new mass downloading application has appeared on my server, > already causing tile requests in the order of millions with up to 600000 > tiles for a single client. The user agent is "Profile/MIDP-2.0 > Configuration/CLDC-1.1" which indicates some sort of handy software, but a > search in the net brought up no clues as to what exactly this might be. Do > you have any suggestions? > I too am perplexed by the MIDP user-agent. I believe the user-agent is from Java Mobile Edition. Some? BlackBerry's prefix their user-agent eg: "BlackBerry9800/6.0.0.246 Profile/MIDP-2.0 Configuration/CLDC-1.1". Basic tile.osm.org user-agent stats: http://grant.dev.openstreetmap.org/tmp/logs/yevaud-user-agents-16042011.txt.bz2 Regards Grant _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
