Hello, is it possible to get a new option to disable the usage of robots.txt (--norobots)?
for example: I want to mirror some parts of http://ftp.de.debian.org/debian/ but the admin have a robots.txt http://ftp.de.debian.org/robots.txt User-agent: * Disallow: / I think he want to protect his machine from searchengine spiders and not from users want to download files.:) it would be great if I could use wget for this task but now its not possible.:( Thanks alot. -- No�l K�the
