Marked OT. On 7/14/05, Eric J. Pastoor <[EMAIL PROTECTED]> wrote: > The msn bot has been hammering one of my webapps for about 2 weeks now. I > have a calendar style transaction page that is not password protected. > Essentially this calendar goes on forever. This means that the msn bot is > basically going on forever. It is making my log files insanely large. Is > there a way to create a robots.txt file for basically one servlet instead of > everything?
In theory, yes, by using the disallow directive appropriately. For example, this snippet recommends all bots stay away from your CGI scripts. User-agent: * Disallow: /cgi-bin/ and this one recommends that badbot stays away from the GoodServlet User-agent: badbot Disallow: /servlets/GoodServlet -Rahul --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
