Hi Hugo
You 're not the only one who has suffered this from Google, but Baidu is
worse and some others as well, giving you telegram answers to your points...
Yes I have also suffered a lot from crawlers, and I have spend a lot of
hours trying to adjut firewalls, robots....
What version of Koha you're using? moderns one have a command
koha-sitemap (If I am not wrong)
I'm on Koha 16.11.04, and yes, there is a command "koha-sitemap".
Unfortunately I couldn't find anything about it in the the Koha 16.11
manual. The only sources I found was:
* https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11190
*
http://search.cpan.org/~fredericd/Koha-Contrib-Tamil-0.011/bin/koha-sitemap
Since I couldn't find the command in
https://wiki.koha-community.org/wiki/Commands_provided_by_the_Debian_packages
I added it there in the new section "Bot-related". But I think it'd be
good to propagate this better - or even activate it by default because
who would want such behavior?
Google Webmaster warns you that it has not inmediate effect, you should
wait a little more...
In summary you have done all the expected work, now it is just time to
ajust it and wait for the results
With the combinations of robots.txt, koha-sitemap & firewall I have been
happy for a long time... but you're not save from this never
:( I am sorry..
I'm getting more patient now since I was at least able to cure the
symptoms...
Best wishes & thanks again: Michael
--
Geschäftsführer · Diplombibliothekar BBS, Informatiker eidg. Fachausweis
Admin Kuhn GmbH · Pappelstrasse 20 · 4123 Allschwil · Schweiz
T 0041 (0)61 261 55 61 · E m...@adminkuhn.ch · W www.adminkuhn.ch
_______________________________________________
Koha mailing list http://koha-community.org
Koha@lists.katipo.co.nz
https://lists.katipo.co.nz/mailman/listinfo/koha
