If I may add (what Gerald is saying) that encompasses any password protected pages. For other pages, if you don't have a robots text file specified, the SpamBots (and search engines) will spider the pages. So, if you don't have one and the page is not password protected, it will get spidered. Now if someone could only come up with a robots text file command that would keep out the parasites and let the search engines in, that would be a miracle.
Are you saying Jeff that this email address IS encoded and it's still getting attacked? -Clint God Bless Us All Clint Hamilton, Owner http://OrpheusComputing.com ) ----- Original Message ----- From: "Gerry Boyd" <[EMAIL PROTECTED]> At 08:37 PM 10/17/03 -0400, [EMAIL PROTECTED] wrote the following: >These places that spider your site, do they just hit the home page or also >hit subpages? >I've had my email addresses encoded for some time now, but the SPA* is >getting a lot worse at the one >email that I only use on the webpages... All pages of all depths and all directories UNLESS one of two conditions are found. 1. Robots.txt file that tells the web spiders what directories are OK or not OK 2. Directories where the Unix file permissions are set to not allow reading Example: Try to see http://www.expita.com/cgi-bin/ whereas this one is open http://www.expita.com/images/ Robots.txt tutorial http://www.searchengineworld.com/robots/robots_tutorial.htm ============= PCWorks Mailing List ================= Don't see your post? Check our posting guidelines & make sure you've followed proper posting procedures, http://pcworkers.com/rules.htm Contact list owner <[EMAIL PROTECTED]> Unsubscribing and other changes: http://pcworkers.com =====================================================
