Hi again,
I don't think that non-existing robots.txt-files should be reported as
broken links (as long as they are not referenced by some page).
Current output, if spanning over 2 hosts (e.g., -D
www.domain1.com,www.domain2.com):
-
Found 2 broken links.
Stefan Melbinger ha scritto:
I don't think that non-existing robots.txt-files should be reported as
broken links (as long as they are not referenced by some page).
Current output, if spanning over 2 hosts (e.g., -D
www.domain1.com,www.domain2.com):
-
Found 2 broken links.
