Came across a website that rejects wget access even with the robots=off option and the user-agent opion being used. It downloads the file index1.html file and stops. The index1.html file contains the following text message:
Sorry, but the download manager you are using to view this site is not supported. We do not support use of such download managers as flashget, go!zilla, or getright. Here the command line I used to access it. Before you try this URL be warned that it has adult content. ;-) wget --user-agent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)" -o /tmp/log --verbose --directory-prefix=/home/Administrator/temp -e robots=off -mirror http://hyapatia69.web1000.com/english/index1.html The site is accessible using Netscape browser. -earlm
