thiemowmde added a comment.

As you said, Special:GoToLinkedPage redirects and does not output HTML (except for the form, which is a single page, and the reason why Disallow: /wiki/Special:GoToLinkedPage should not be used). The target pages of these redirects are Wikipedia articles. They should be indexed, and they already have canonical tags.

I'm not sure what an X-Robots-Tag will do when used with a redirect.

  • I believe there is no point in crawling Special:GoToLinkedPage URLs, because they are guaranteed to do nothing but redirect to Wikipedia articles. We know each redirect represents a sitelink, and sitelinks are already accessible on the ordinary item page. We know all this. Google does not.
  • Similar for Special:ItemByTitle, which redirects to a Wikidata item. The exact same links already exist in the sidebars of the connected Wikipedia articles.

@Mbch331, please add the following lines to

Disallow: /wiki/Special:GoToLinkedPage?
Disallow: /wiki/Special:ItemByTitle?
Disallow: /wiki/Special:SetSiteLink?

Do not remove the slashes, because this would exclude the special page forms itself. We want these to appear in a Google search.



To: Mbch331, thiemowmde
Cc: Sjoerddebruin, TheDJ, Mbch331, Jonas, hoo, aude, Lydia_Pintscher, Tobi_WMDE_SW, Aklapper, thiemowmde, TerraCodes, D3r1ck01, MuhammadShuaib, Izno, Wikidata-bugs
Wikidata-bugs mailing list

Reply via email to