On Sun, Aug 15, 2010 at 3:03 PM, Peter Körner <[email protected]> wrote: > Am 15.08.2010 18:39, schrieb Anthony: >> >> I see http://planet.openstreetmap.org/robots.txt now has User-agent: * >> and Disallow: / >> >> Are we allowed to download the minute-replicate files as they become >> available? If not, what's the point of having them? > > Are you a robot?
No, but my perl script is. On Sun, Aug 15, 2010 at 3:12 PM, Nic Roets <[email protected]> wrote: > Hello Anthony, > > AFAIK, robots.txt only applies to recursive downloads. Given that file > names follow simple patterns and timestamp files exist, it is really > not necessary to run recursive spiders. Thanks. After this explanation and a quick reading of http://www.robotstxt.org/faq/what.html I'm going to conclude that my perl script is not restricted by robots.txt. _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
