Hello,
When I first came across this I was sure it was a bug. Now that I have
thought about it a bit more, I am not so sure. It relates to HTTP
redirection, -L, and spanning hosts.
I have some sites with pages that set the location field in the header to
force a redirection. These redirects are not to "local" or relative URLs.
I tried to mirror one of these sites with wget -r -L and wget -m -L.
What I found is that wget went out to those sites that the redirects
pointed to.
For starters, I thought that the default behaviour of wget was NOT to span
hosts. If this is so then why did it follow the redirects and get stuff
from other hosts? Because of the above I tried using -L which is supposed
to keep wget from following non-relative links. I suppose a redirect is
not a "link" but it is a URL. Needless to say, -L did not fix the problem
of taking offsite redirects.
Two questions. Firstly, it is a bug that neither the default behaviour of
NOT spanning hosts and -L won't stop wget from following an offsite
rediect? Secondly, if it is not a bug then how does one stop wget from
taking an offsite rediect.
Thanks,
Justin DeVuyst
