Hi Ercolino,
This is due to --spider issuing a GET request instead of a HEAD request.
I submitted a patch on 7th August for this. Details and patch copied here:
In GNU wget[1], `--spider' first issues a HEAD request, then iff HEAD fails,
issues a GET request. In Busybox wget, only a GET request is sent. According to
the HTTP spec[2], Busybox wget impl is technically correct. However, it causes
portability problems. Interestingly, GNU wget man page[3] does not explicitly
specify HTTP method.
Attached is patch to fix. Note: this is still not totally compatible with GNU
wget because it does not retry with GET if HEAD fails.
Thank you,
Jake
[1] https://git.savannah.gnu.org/cgit/wget.git/tree/src/http.c#n4304
[2] https://httpwg.org/specs/rfc7231.html#HEAD
[3] https://www.gnu.org/software/wget/manual/wget.html#index-spider
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Thursday, September 30th, 2021 at 8:59 AM, Ercolino de Spiacico
<[email protected]> wrote:
> Thanks Nicholas
>
> I sort of got to the same conclusion that -S is somehow referring to
>
> --spider or a sort of alias to it.
>
> There is one thing that puzzles me though, the -S will show the Server
>
> response indeed but also download the file. This latter action defeats
>
> the concept for my purpose (check RAM availability prior to download to
>
> prevent starvation).
>
> Say you're in my situation, with embedded-system's RAM overloaded you
>
> need to make a decision if a file download will fit or not, -S will
>
> provide the info but potentially already creating the damage.
>
> Is there any way to get the file size only without downloading first?
>
> I've tried appending "-O /dev/null", somehow it seems to work (file not
>
> created) but I'm not sure this is the right approach... Not triggering
>
> the file download at all would be the optimal result I suppose.
>
> busybox mailing list
>
> [email protected]
>
> http://lists.busybox.net/mailman/listinfo/busybox
diff --git a/networking/wget.c b/networking/wget.c
index 6a9604421..93c995205 100644
--- a/networking/wget.c
+++ b/networking/wget.c
@@ -1213,7 +1213,8 @@ static void download_one_url(const char *url)
target.path);
} else {
SENDFMT(sfp, "%s /%s HTTP/1.1\r\n",
- (option_mask32 & WGET_OPT_POST_DATA) ? "POST" : "GET",
+ (option_mask32 & WGET_OPT_POST_DATA) ? "POST" :
+ ((option_mask32 & WGET_OPT_SPIDER) ? "HEAD" : "GET"),
target.path);
}
if (!USR_HEADER_HOST)
_______________________________________________
busybox mailing list
[email protected]
http://lists.busybox.net/mailman/listinfo/busybox
