At 11:22 AM 2/27/2002, you wrote: >Adam, > > > hmm - wouldn't that also block search spiders as well? I can imagine they > > spider fairly quickly throughout your site. This may be your desired > > functionality however. > >You are right, I guess I could add a simple filter using CGI USER AGENT >and sort out fast/altavista/google and other search engines I know about >(and care about).
Spiderhunter.com has data available on the IP's useragents and DNS of spiders around the place, I've used it to do spider detection work in the past (and may do so in the future too when I get around to it :-)). http://www.spiderhunter.com/spiderlist/index.cfm?FuseAction=Results&Key=1470.26536188&SearchEngineIDLink=-1&ReportFormat=1&information=2 James Sleeman Innovative Media Ltd Phone: (03) 377 6262 http://www.websolutions.co.nz/ CAUTION: The information contained in this email message is confidential and may be legally privileged. If the reader of this message is not the intended recipient you are notified that any use, dissemination, distribution or reproduction of this message is prohibited. If you have received this message in error please notify the sender immediately and destroy the original message and any attachments. Views expressed in this communication may not be those of Innovative Media Ltd. ______________________________________________________________________ Dedicated Windows 2000 Server PIII 800 / 256 MB RAM / 40 GB HD / 20 GB MO/XFER Instant Activation � $99/Month � Free Setup http://www.pennyhost.com/redirect.cfm?adcode=coldfusiona FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Archives: http://www.mail-archive.com/[email protected]/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
