man wget shows:
-D domain-list
--domains=domain-list
Set domains to be followed. domain-list is a comma-separated
list of domains.
Note that it does not turn on -H.
Mark Post
-----Original Message-----
From: Andrzej Kasperowicz [mailto:[EMAIL PROTECTED]
Sent: Monday, August 18, 2003 8:38 AM
To: [EMAIL PROTECTED]
Subject: wget is mirroring whole internet instead of just my web page!
When I try to mirror web pages using the command:
wget -m -nv -k -K -nH -t 100 -o logchemfanpl -P public_html/mirror
http://znik.wbc.lublin.pl/ChemFan/
wget is mirroring not just the domain of the web page but just whole
internet...
There is robot.txt files, but it should not influence wget to
download all available domains I suppose?
So why is it happening and how to avoid it?
Regards
Andrzej.
