This is an automated email from the git hooks/post-receive script.
git pushed a commit to branch master
in repository www.
View the commit online.
commit 21a587463b1aa8cff2d398219e8df215c455cc40
Author: Carsten Haitzler (Rasterman) <ras...@rasterman.com>
AuthorDate: Thu Sep 26 18:47:25 2024 +0100
expand robots.txt to a list of explicit bots only
---
robots.txt | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/robots.txt b/robots.txt
index c2a49f4f..50ac5aff 100644
--- a/robots.txt
+++ b/robots.txt
@@ -1,2 +1,13 @@
-User-agent: *
+User-agent: Googlebot
+User-agent: bingbot
+User-agent: Baiduspider
+User-agent: DuckDuckBot
+User-agent: YandexBot
+User-agent: naver
+User-agent: Applebot
+User-agent: OAI-SearchBot
+User-agent: GPTBot
Allow: /
+
+User-agent: *
+Disallow: /
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
