Only if you use the Verity spider (CF version 5.0 and greater). By default it will honor any robots.txt files that are there. There is a switch you can specify (-norobo) to tell it to ignore the robots.txt directives.
Jeff -----Original Message----- From: Kwang Suh [mailto:[EMAIL PROTECTED]] Sent: Monday, July 08, 2002 10:20 PM To: CF-Talk Subject: RE: Selective Verity? Only sort of related, but does Verity follow directives in robots.txt? > -----Original Message----- > From: Dave Watts [mailto:[EMAIL PROTECTED]] > Sent: Monday, July 08, 2002 9:52 PM > To: CF-Talk > Subject: RE: Selective Verity? > > > > I'm not very familiar with Verity. Is there a way to exclude > > a certain directory from being indexed? I need to index most > > of the site but would like to exclude a few private directories. > > There's a CFML custom tag on the MM Developers Exchange that will > allow you > to do this, by specifying the directories that you want to exclude: > > http://devex.macromedia.com/developer/gallery/info.cfm?ID=A155CD89 -7445-11D4 -AAA400508B94F380&method=Full Dave Watts, CTO, Fig Leaf Software http://www.figleaf.com/ voice: (202) 797-5496 fax: (202) 797-5444 ______________________________________________________________________ Structure your ColdFusion code with Fusebox. Get the official book at http://www.fusionauthority.com/bkinfo.cfm FAQ: http://www.thenetprofits.co.uk/coldfusion/faq Archives: http://www.mail-archive.com/[email protected]/ Unsubscribe: http://www.houseoffusion.com/index.cfm?sidebar=lists
