On 18.06.21 18:22, Stephane Ascoet wrote:
I've solved it: the directory is forbidden by robots.txt
Wget should mention this on standard input when finished, something like
"some elements couldn't be downloaded because of robots.txt rules"
Good suggestion ! Maybe even with the exact number of
I've solved it: the directory is forbidden by robots.txt
Wget should mention this on standard input when finished, something like
"some elements couldn't be downloaded because of robots.txt rules"
--
Sincerely, Stephane Ascoet
Hi, I use Wget since years for basic individuals download. I knew that
it was able to mirror Website but never had the need for this. This
changed a few months ago, and by reading the documentation and making
lots of tests and retries, I succeeded in obtaining the results I was
expecting. Wget
