https://bugzilla.wikimedia.org/show_bug.cgi?id=44731

--- Comment #12 from Matthew Flaschen <[email protected]> ---
(In reply to Matthew Flaschen from comment #11)
> (In reply to Jan Zerebecki from comment #10)
> > Maybe the link doesn't get updated because robots.txt has Disallow:
> > /pipermail/ . Perhaps making http://mail.wikipedia.org/robots.txt (only for
> > that domain) a 404 has the desirable effect.
> 
> I think that may have been the original domain, so that will probably still
> lead to crawling the archives.

Actually never mind, as long as the crawlers respect the destination robots.txt
(after being redirected) it should be fine.

-- 
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l

Reply via email to