I've been blocking librabot if it appears in the user-agent. Info online is sketchy about whether or not librabot is bad; one person claimed that it did not honor robots.txt, and another at Microsoft said that it wasn't anything of theirs, in spite of the Microsoft reference (see request snippet below). The connecting addresses often don't resolve at all (i.e., no PTR records in DNS).
So is there any consensus on whether or not it should be blocked? My conclusion so far is that I should continue to block it. Part of request: From: [email protected] User-Agent: librabot/2.0 (+http://academic.research.microsoft.com/) -- Shane Castle Data Security Mgr, Boulder County IT CISSP GSEC GCIH _______________________________________________ Owasp-modsecurity-core-rule-set mailing list [email protected] https://lists.owasp.org/mailman/listinfo/owasp-modsecurity-core-rule-set
