Hi Yes this is annoying issue with boots, this is google but there are plenty of them...
You should use robots.txt propertly, but If I am not wrong with Google it is more effective go to google webmaster web and modify the googleboot behaviour with your koha installarion You should also use a koha-sitemap.. depending on the version is out of the box functionality Perhaps you may think on use ufw or even ufw + fail2ban Some times bots are nightmare 2017-05-03 13:49 GMT+02:00 Mark Alexander <ma...@pobox.com>: > > When I searched for who is 66.249.64.32 I saw this IP addresse belongs > > to Google. > > This does seem to be the Google indexer: > > % nslookup 66.249.64.32 > ... > 32.64.249.66.in-addr.arpa name = crawl-66-249-64-32.googlebot.com. > > I haven't seen this problem (yet), but perhaps that is because I have > a /usr/share/koha/opac/htdocs/robots.txt containing this: > > Crawl-delay: 60 > > User-agent: * > Disallow: / > > User-agent: Googlebot > Disallow: /cgi-bin/koha/opac-search.pl > Disallow: /cgi-bin/koha/opac-showmarc.pl > Disallow: /cgi-bin/koha/opac-detailprint.pl > Disallow: /cgi-bin/koha/opac-ISBDdetail.pl > Disallow: /cgi-bin/koha/opac-MARCdetail.pl > Disallow: /cgi-bin/koha/opac-reserve.pl > Disallow: /cgi-bin/koha/opac-export.pl > Disallow: /cgi-bin/koha/opac-detail.pl > Disallow: /cgi-bin/koha/opac-authoritiesdetail.pl > _______________________________________________ > Koha mailing list http://koha-community.org > Koha@lists.katipo.co.nz > https://lists.katipo.co.nz/mailman/listinfo/koha > -- *Hugo Agud - Orex Digital * *www.orex.es <http://www.orex.es>* <http://www.orex.es/> [image: www.orex.es/koha] <http://www.orex.es/koha> [image: www.orex.es/vufind] <http://www.orex.es/vufind> <http://www.orex.es/omeka> Director Calle Sant Joaquin,117, 2º-3ª · 08922 Santa Coloma de Gramanet - Tel: 933 856 138 ha...@orex.es · http://www.orex.es/ No imprima este mensaje a no ser que sea necesario. Una tonelada de papel implica la tala de 15 árboles y el consumo de 250.000 litros de agua. Aviso de confidencialidad Este mensaje contiene información que puede ser CONFIDENCIAL y/o de USO RESTRINGIDO. Si usted no es el receptor deseado del mensaje (ni está autorizado a recibirlo por el remitente), no está autorizado a copiar, reenviar o divulgar el mensaje o su contenido. Si ha recibido este mensaje por error, por favor, notifíquenoslo inmediatamente y bórrelo de su sistema. _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha