Merlijn van Deen has uploaded a new change for review. https://gerrit.wikimedia.org/r/287249
Change subject: Add crawl delay ...................................................................... Add crawl delay Change-Id: I9ee48c1ed9f836cde86e263fee2385690b9c121e --- M www/robots.txt 1 file changed, 1 insertion(+), 0 deletions(-) git pull ssh://gerrit.wikimedia.org:29418/labs/toollabs refs/changes/49/287249/1 diff --git a/www/robots.txt b/www/robots.txt index c1b1bb8..5c52907 100644 --- a/www/robots.txt +++ b/www/robots.txt @@ -1,4 +1,5 @@ User-agent: * +Crawl-delay: 3 Disallow: /betacommand-dev/ Disallow: /fiwiki-tools/ -- To view, visit https://gerrit.wikimedia.org/r/287249 To unsubscribe, visit https://gerrit.wikimedia.org/r/settings Gerrit-MessageType: newchange Gerrit-Change-Id: I9ee48c1ed9f836cde86e263fee2385690b9c121e Gerrit-PatchSet: 1 Gerrit-Project: labs/toollabs Gerrit-Branch: master Gerrit-Owner: Merlijn van Deen <[email protected]> _______________________________________________ MediaWiki-commits mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
