Ok, either I've completely misread wget, or it has a problem
mirroring SSL sites. It appears that it is deciding that
the https:// scheme is something that is "not to be followed".
For those interested, the offending code appears to be 3 lines
in recur.c, which, if changed treat the HTTPS schema the same
way that the HTTP schema is treated with respect to following
links and in-line content:
Line 440: change to
if (u->scheme != SCHEME_HTTP && u->scheme!= SCHEME_HTTPS
Line 449: change to
if (u->scheme == SCHEME_HTTP || u->scheme == SCHEME_HTTPS)
Line 537: change to
if (opt.use_robots && (u->scheme == SCHEME_HTTP || u->scheme ==
SCHEME_HTTPS))
For those wanting a patch, one that can be applied to the 1.8.1
source distribution is attached.
Thomas
--- src/recur.c Wed Dec 19 09:27:29 2001
+++ ../wget-1.8.1.esoft/src/recur.c Sat Dec 29 16:17:40 2001
@@ -437,7 +437,7 @@
the list. */
/* 1. Schemes other than HTTP are normally not recursed into. */
- if (u->scheme != SCHEME_HTTP
+ if (u->scheme != SCHEME_HTTP && u->scheme!= SCHEME_HTTPS
&& !(u->scheme == SCHEME_FTP && opt.follow_ftp))
{
DEBUGP (("Not following non-HTTP schemes.\n"));
@@ -446,7 +446,7 @@
/* 2. If it is an absolute link and they are not followed, throw it
out. */
- if (u->scheme == SCHEME_HTTP)
+ if (u->scheme == SCHEME_HTTP || u->scheme == SCHEME_HTTPS)
if (opt.relative_only && !upos->link_relative_p)
{
DEBUGP (("It doesn't really look like a relative link.\n"));
@@ -534,7 +534,7 @@
}
/* 8. */
- if (opt.use_robots && u->scheme == SCHEME_HTTP)
+ if (opt.use_robots && (u->scheme == SCHEME_HTTP || u->scheme == SCHEME_HTTPS))
{
struct robot_specs *specs = res_get_specs (u->host, u->port);
if (!specs)
2001-12-29 Thomas Reinke <[EMAIL PROTECTED]>
* recur.c: fixed scheme handling for https to allow proper following
of links
