https://bugzilla.wikimedia.org/show_bug.cgi?id=65464
Bug ID: 65464
Summary: bits.wikimedia.org discourages indexing, resulting in
goofy archive.org snapshots
Product: Wikimedia
Version: wmf-deployment
Hardware: All
OS: All
Status: NEW
Keywords: ops
Severity: normal
Priority: Unprioritized
Component: General/Unknown
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected]
Web browser: ---
Mobile Platform: ---
bits.wikimedia.org was set up to serve JavaScript and CSS in January 2010 (cf.
[[wikitech:Server admin log/Archive 15#January 10]].
Its [[robots.txt]] file disallows everything:
$ curl bits.wikimedia.org/robots.txt
User-agent: *
Disallow: /
Comparing the French Wikipedia main page from 2009
(<https://web.archive.org/web/20090601000000*/http://fr.wikipedia.org/wiki/Accueil>)
to 2010
(<https://web.archive.org/web/20110101000000*/http://fr.wikipedia.org/wiki/Accueil>)
shows its effect: the generated snapshots look goofy.
Perhaps we should specifically re-enable the IA bot?
This issue was reported by ytrezq in #wikimedia-operations in freenode.
--
You are receiving this mail because:
You are the assignee for the bug.
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
