It would seem that this is not your site then and so if the owner of the 
site doesn't want you to download his/her entire site why should you?

Maybe they have banners or something that pays for their hosting costs 
and they feel preventing tools like wget form sucking their site is a 
way to protect themselves.

On Friday, September 13, 2002, at 11:14 AM, Earl Mitchell wrote:

> Came across a website that rejects wget
> access even with the robots=off option and the user-agent
> opion being used.  It downloads the file index1.html file and
> stops. The index1.html file contains the following text message:
>
> Sorry, but the download manager you are using to view this site is not 
> supported.  We do not support use of such download managers as 
> flashget, go!zilla, or getright.
>
> Here the command line I used to access it. Before you try this
> URL be warned that it has adult content. ;-)
>
> wget --user-agent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)"  
> -o /tmp/log --verbose --directory-prefix=/home/Administrator/temp -e 
> robots=off -mirror http://hyapatia69.web1000.com/english/index1.html
>
> The site is accessible using Netscape browser.
>
> -earlm

Reply via email to