Try here: http://www.robotstxt.org/wc/robots.html
-----Original Message----- From: Scott Purcell [mailto:[EMAIL PROTECTED] Sent: Wednesday, March 22, 2006 8:00 PM To: Tomcat Users List Subject: Would like to track googlebots, or spiders from site Not necessarily a tomcat problem, but I have created a webapp on Tomcat 5.x. I would like to be able to track any robot activity in a log file, but not really sure where to begin looking for this functionality? If this is possible, is there a debug level that could be used so I could find out what pages it hits, etc? Or is this something that needs to be written and tired into the container. This is Tomcat standalone, no Apache front end. If anyone has this knowledge, links, etc. I would certainly appreciate. Regards Scott --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
