Brett, How effective is this method, do you just keep adding or will it catch 98% of bots?
Regards Andrew Scott Analyst Programmer CMS Transport Systems Level 2/33 Bank Street South Melbourne, Victoria, 3205 Phone: 03 9699 7988 - Fax: 03 9699 7976 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brett Payne-Rhodes Sent: Wednesday, 13 July 2005 12:38 PM To: CFAussie Mailing List Subject: [cfaussie] RE: Search Engines This is what I use to try and identify 'bots' <!--- Check browser for 'bot' identifiers... ---> <cfset botList = "bot,crawler,ia_archiver,pompos,WebRescuer,/ysearch/,ZyBorg"> <cfloop list="#botList#" index="botName"> <cfif (findNoCase(botName, cgi.http_user_agent) gt 0)> <cfset session.botSession = true> <cfbreak> </cfif> </cfloop> No warranty or guarantee implied... ;) B) Andrew Scott wrote: > Guys, > > Please I am not interested in the ins and outs of search engines. I really > need this list for the project and that's it. > > I have been trying to solve this problem for now 6 months, and the > robots.txt file is correct, and I need to stop them from accessing content > in my code and my only option due to the way this website has been written > is to see if the search engine has arrived and stop them from displaying > certain content in a page. > > > Regards > Andrew Scott > Analyst Programmer > > CMS Transport Systems > Level 2/33 Bank Street > South Melbourne, Victoria, 3205 > > Phone: 03 9699 7988 - Fax: 03 9699 7976 > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dawes, Chris > Sent: Wednesday, 13 July 2005 12:16 PM > To: CFAussie Mailing List > Subject: [cfaussie] RE: Search Engines > > > Have you set the file up correctly? > > http://www.searchengineworld.com/cgi-bin/robotcheck.cgi > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Andrew > Scott > Sent: Wednesday, 13 July 2005 12:16 PM > To: CFAussie Mailing List > Subject: [cfaussie] RE: Search Engines > > I have that as well, but they are ignoring this file. > > Regards > Andrew Scott > Analyst Programmer > > CMS Transport Systems > Level 2/33 Bank Street > South Melbourne, Victoria, 3205 > > Phone: 03 9699 7988 - Fax: 03 9699 7976 > > -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of Dawes, > Chris > Sent: Wednesday, 13 July 2005 12:07 PM > To: CFAussie Mailing List > Subject: [cfaussie] RE: Search Engines > > > Once again robots.txt is your friend... It tells the spiders where to > look and where NOT to look... > > Dawesi > > > > > --- > You are currently subscribed to cfaussie as: [EMAIL PROTECTED] > To unsubscribe send a blank email to > [EMAIL PROTECTED] > Aussie Macromedia Developers: http://lists.daemon.com.au/ > > ---------------------------------------------------- > This email and any files transmitted with it are confidential and intended > solely for the use of the individual or entity to whom they are addressed. > If you have received this email in error please notify the system manager. > Please note that any views or opinions presented in this email are solely > those of the author and do not necessarily represent those of the company. > The recipient should check this email and any attachments for the presence > of viruses. The company accepts no liability for any damage caused by any > virus transmitted by this email. > ---------------------------------------------------- > > --- > You are currently subscribed to cfaussie as: [EMAIL PROTECTED] > To unsubscribe send a blank email to > [EMAIL PROTECTED] > Aussie Macromedia Developers: http://lists.daemon.com.au/ > > > > > --- > You are currently subscribed to cfaussie as: [EMAIL PROTECTED] > To unsubscribe send a blank email to [EMAIL PROTECTED] > Aussie Macromedia Developers: http://lists.daemon.com.au/ > --- You are currently subscribed to cfaussie as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] Aussie Macromedia Developers: http://lists.daemon.com.au/ --- You are currently subscribed to cfaussie as: [email protected] To unsubscribe send a blank email to [EMAIL PROTECTED] Aussie Macromedia Developers: http://lists.daemon.com.au/
