I have heard much talk of the security issue recently. Here is one way to get around this. Lets say you have a file called "private". You could put it in a folder called "icons" (or some other generic name) and then do a disallow on "/icons" that way you wouldn't be giving away the name of your private directory. Here is an example:
the file that you want to keep private = "private" place "private" in the following path: /icons/private then make your robots.txt file read as such: User-agent: * Disallow: /icons If anyone has any feedback on this, please let me know. Thanks, Anthony Kirlew Nocturnal Solutions Web Design / Search Engine Optimization Specialists www.nocturnalsolutions.net [EMAIL PROTECTED] ----- Original Message ----- From: Klaus Johannes Rusch <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Sent: Thursday, March 09, 2000 12:53 PM Subject: Re: robots.txt a security hole?? > In <[EMAIL PROTECTED]>, Andrew Daviel <[EMAIL PROTECTED]> writes: > > Some Web Servers use a file called /robot(s).txt to make search engines > > and any other indexing tools visit their WebPages more frequently and more > > efficiently. By connecting to the server and requesting the /robot(s).txt > > file, an attacker may gain additional information about the system they > > are attacking. Such information as, restricted directories, hidden > > directories, cgi script directories and etc. Take special care not to tell > > the robots not to index sensitive directories, since this tells attackers > > exactly which of your directories are sensitive. > > The risk is webmasters not protecting sensitive resources properly but only > trying to obsure them. > > robots.txt is not a place for listing sensitive directories but for helping > robots avoid indexing irrelevant information or going into endless loops on > dynamic pages. > > There has been some discussions in comp.risks in 1998 which is summarized at > http://www.eiffel.com/private/meyer/robots.html > > Klaus Johannes Rusch > -- > [EMAIL PROTECTED] > http://www.atmedia.net/KlausRusch/
