20after4 has uploaded a new change for review.
https://gerrit.wikimedia.org/r/234601
Change subject: Add a bunch of apps and expensive URLs to robots.txt exclusion
......................................................................
Add a bunch of apps and expensive URLs to robots.txt exclusion
Bug: T110710
Change-Id: I67623e657feb18c895bdbd1e17e49d84c8255138
---
M src/applications/system/controller/PhabricatorRobotsController.php
1 file changed, 16 insertions(+), 0 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/phabricator/phabricator
refs/changes/01/234601/1
diff --git a/src/applications/system/controller/PhabricatorRobotsController.php
b/src/applications/system/controller/PhabricatorRobotsController.php
index 11177f4..5ac822e 100644
--- a/src/applications/system/controller/PhabricatorRobotsController.php
+++ b/src/applications/system/controller/PhabricatorRobotsController.php
@@ -22,6 +22,22 @@
$out[] = 'Disallow: /diffusion/';
$out[] = 'Disallow: /multimeter/';
$out[] = 'Disallow: /file/';
+ $out[] = 'Disallow: /project/sprint';
+ $out[] = 'Disallow: /policy/explain';
+ $out[] = 'Disallow: /auth';
+ $out[] = 'Disallow: /login';
+ $out[] = 'Disallow: /maniphest/transaction';
+ $out[] = 'Disallow: /tag';
+ $out[] = 'Disallow: /search/query/all';
+ $out[] = 'Disallow: /conduit';
+ $out[] = 'Disallow: /api';
+ $out[] = 'Disallow: /project';
+ $out[] = 'Disallow: /applications';
+ $out[] = 'Disallow: /token';
+ $out[] = 'Disallow: /pholio';
+ $out[] = 'Disallow: /dashboard';
+ $out[] = 'Disallow: /calendar';
+ $out[] = 'Disallow: /herald';
// Add a small crawl delay (number of seconds between requests) for spiders
// which respect it. The intent here is to prevent spiders from affecting
--
To view, visit https://gerrit.wikimedia.org/r/234601
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I67623e657feb18c895bdbd1e17e49d84c8255138
Gerrit-PatchSet: 1
Gerrit-Project: phabricator/phabricator
Gerrit-Branch: production
Gerrit-Owner: 20after4 <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
