Well Andrew, I provided both a way to stop them coming in _AND_ a way to find a list of spiders.
If you all you want is the spider list, type robots.txt into Google and whazam... Well lookie here lots of pages with lists of spiders. (c: That's because spiders use robots.txt... A bit lateral for some, but it'll give you what you want. For another $50 I can print it and mail it to you too. ;-) Dawesi -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Scott Sent: Wednesday, 13 July 2005 12:26 PM To: CFAussie Mailing List Subject: [cfaussie] RE: Search Engines Guys, Please I am not interested in the ins and outs of search engines. I really need this list for the project and that's it. I have been trying to solve this problem for now 6 months, and the robots.txt file is correct, and I need to stop them from accessing content in my code and my only option due to the way this website has been written is to see if the search engine has arrived and stop them from displaying certain content in a page. Regards Andrew Scott Analyst Programmer CMS Transport Systems Level 2/33 Bank Street South Melbourne, Victoria, 3205 Phone: 03 9699 7988 - Fax: 03 9699 7976 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dawes, Chris Sent: Wednesday, 13 July 2005 12:16 PM To: CFAussie Mailing List Subject: [cfaussie] RE: Search Engines Have you set the file up correctly? http://www.searchengineworld.com/cgi-bin/robotcheck.cgi -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andrew Scott Sent: Wednesday, 13 July 2005 12:16 PM To: CFAussie Mailing List Subject: [cfaussie] RE: Search Engines I have that as well, but they are ignoring this file. Regards Andrew Scott Analyst Programmer CMS Transport Systems Level 2/33 Bank Street South Melbourne, Victoria, 3205 Phone: 03 9699 7988 - Fax: 03 9699 7976 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dawes, Chris Sent: Wednesday, 13 July 2005 12:07 PM To: CFAussie Mailing List Subject: [cfaussie] RE: Search Engines Once again robots.txt is your friend... It tells the spiders where to look and where NOT to look... Dawesi --- You are currently subscribed to cfaussie as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] Aussie Macromedia Developers: http://lists.daemon.com.au/ ---------------------------------------------------- This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. ---------------------------------------------------- --- You are currently subscribed to cfaussie as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] Aussie Macromedia Developers: http://lists.daemon.com.au/ --- You are currently subscribed to cfaussie as: [EMAIL PROTECTED] To unsubscribe send a blank email to [EMAIL PROTECTED] Aussie Macromedia Developers: http://lists.daemon.com.au/ ---------------------------------------------------- This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. Please note that any views or opinions presented in this email are solely those of the author and do not necessarily represent those of the company. The recipient should check this email and any attachments for the presence of viruses. The company accepts no liability for any damage caused by any virus transmitted by this email. ---------------------------------------------------- --- You are currently subscribed to cfaussie as: [email protected] To unsubscribe send a blank email to [EMAIL PROTECTED] Aussie Macromedia Developers: http://lists.daemon.com.au/
