--- Bob <[EMAIL PROTECTED]> wrote: > I've had someone probing for "/cgi-bin/****" in my error log. > They got a 403 error as it's a protected directory, but I'd put it in my > "robots.txt" file, as most examples suggest. > > Should I take this out, as the robots file is available for anyone to read, > or is it needed? > Thanks.
/cgi-bin is such a common directory name that a malicious user could attempt to read files from it whether it is in your robots.txt file or not. I recall hearing about a large company (Kodak.com?) which had noindex options on certain directories and when people looked at the file, they discovered that the directories contained sensitive and unprotected data. The purpose of the robots.txt file is to guide search engine spiders. The spiders should not even try to look into these directories. As you noted, on a well-configured system the original source code files will not be revealed. James ------------------------ Yahoo! Groups Sponsor --------------------~--> Most low income households are not online. Help bridge the digital divide today! http://us.click.yahoo.com/cd_AJB/QnQLAA/TtwFAA/HKFolB/TM --------------------------------------------------------------------~-> Community email addresses: Post message: [email protected] Subscribe: [EMAIL PROTECTED] Unsubscribe: [EMAIL PROTECTED] List owner: [EMAIL PROTECTED] Shortcut URL to this page: http://groups.yahoo.com/group/php-list Yahoo! Groups Links <*> To visit your group on the web, go to: http://groups.yahoo.com/group/php-list/ <*> To unsubscribe from this group, send an email to: [EMAIL PROTECTED] <*> Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
