On 01/03/14 11:27 AM, Daniel Micay wrote: > The blogs.gnome.org uses a strange robots.txt > <http://blogs.gnome.org/robots.txt> permitting only Googlebot to crawl > it. This prevents using archive.org on the subdomain and cripples other > search engines. It should simply use a wildcard for the first User-agent > field with the second case discarded. > > Thanks!
It looks like this has already been fixed. That could be the best bug report response time I've ever seen. ;)
signature.asc
Description: OpenPGP digital signature
_______________________________________________ gnome-web-list mailing list gnome-web-list@gnome.org https://mail.gnome.org/mailman/listinfo/gnome-web-list
