Hi, On Cygwin, wget 1.16.1 (and current Git master) occasionally fails to fetch release artifacts hosted on GitHub. Example URL:
https://github.com/openslide/openslide/releases/download/v3.4.0/openslide-3.4.0.tar.xz GitHub redirects the request to an Amazon S3 signed URL, and that signature sometimes includes a percent-encoded "+" character. In this case, wget improperly decodes the "%2B" back to "+" before processing the redirect, causing Amazon to return 403. Wget then repeatedly retries the request to GitHub until either 1) the redirect URL does not include a "+", or 2) 20 failed redirects have occurred. This does not happen if IRI is compiled out, does not happen if wget is run from the Cygwin shell, and does not happen on Linux. It does occur if wget is run from the Windows command shell (or, as I originally discovered it, from a Buildbot process running under cygrunsrv). wget version info: C:\cygwin\home\user\wget>src\wget.exe -V GNU Wget 1.16.1.36-01d5 built on cygwin. +digest +https +ipv6 +iri +large-file +nls +ntlm +opie -psl +ssl/gnutls Wgetrc: /usr/local/etc/wgetrc (system) Locale: /usr/local/share/locale Compile: gcc -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc" -DLOCALEDIR="/usr/local/share/locale" -I. -I../lib -I../lib -Iyes/include -I/usr/include/p11-kit-1 -DHAVE_LIBGNUTLS -DNDEBUG Link: gcc -I/usr/include/p11-kit-1 -DHAVE_LIBGNUTLS -DNDEBUG -Lyes/lib -liconv -lintl -lpcre -luuid -lnettle -lgnutls -lz -lintl -liconv -lp11-kit -lgmp -lhogweed -lgmp -lnettle -ltasn1 -lp11-kit -lz -lz -lidn ftp-opie.o gnutls.o http-ntlm.o ../lib/libgnu.a Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://www.gnu.org/licenses/gpl.html>. This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Originally written by Hrvoje Niksic <[email protected]>. Please send bug reports and questions to <[email protected]>. Sample failure: C:\cygwin\home\user\wget>src\wget.exe https://github.com/openslide/openslide/releases/download/v3.4.0/openslide-3.4.0.tar.xz --2015-01-03 06:41:09-- https://github.com/openslide/openslide/releases/download/v3.4.0/openslide-3.4.0.tar.xz Connecting to github.com (github.com)|192.30.252.131|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://s3.amazonaws.com/github-cloud/releases/827644/96a6c9f0-8629-11e3-9b05-739b7f3aed83.xz?response-content-disposition=attachment%3B%20filename%3Dopenslide-3.4.0.tar.xz&response-content-type=application/octet-stream&AWSAccessKeyId=AKIAISTNZFOVBIJMK3TQ&Expires=1420267329&Signature=5d6RHL2oIvoDkET%2BZSiGgT4ZCY0%3D [following] --2015-01-03 06:41:09-- https://s3.amazonaws.com/github-cloud/releases/827644/96a6c9f0-8629-11e3-9b05-739b7f3aed83.xz?response-content-disposition=attachment;%20filename=openslide-3.4.0.tar.xz&response-content-type=application/octet-stream&AWSAccessKeyId=AKIAISTNZFOVBIJMK3TQ&Expires=1420267329&Signature=5d6RHL2oIvoDkET+ZSiGgT4ZCY0= Connecting to s3.amazonaws.com (s3.amazonaws.com)|54.231.2.56|:443... connected. HTTP request sent, awaiting response... 403 Forbidden 2015-01-03 06:41:09 ERROR 403: Forbidden. --2015-01-03 06:41:09-- https://github.com/openslide/openslide/releases/download/v3.4.0/openslide-3.4.0.tar.xz Connecting to github.com (github.com)|192.30.252.131|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://s3.amazonaws.com/github-cloud/releases/827644/96a6c9f0-8629-11e3-9b05-739b7f3aed83.xz?response-content-disposition=attachment%3B%20filename%3Dopenslide-3.4.0.tar.xz&response-content-type=application/octet-stream&AWSAccessKeyId=AKIAISTNZFOVBIJMK3TQ&Expires=1420267330&Signature=MK1DBqMuFnPJ2BF0dOTSnPxDkw0%3D [following] --2015-01-03 06:41:10-- https://s3.amazonaws.com/github-cloud/releases/827644/96a6c9f0-8629-11e3-9b05-739b7f3aed83.xz?response-content-disposition=attachment;%20filename=openslide-3.4.0.tar.xz&response-content-type=application/octet-stream&AWSAccessKeyId=AKIAISTNZFOVBIJMK3TQ&Expires=1420267330&Signature=MK1DBqMuFnPJ2BF0dOTSnPxDkw0= Connecting to s3.amazonaws.com (s3.amazonaws.com)|54.231.2.56|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 401488 (392K) [application/octet-stream] Saving to: 'openslide-3.4.0.tar.xz.2' openslide-3.4.0.tar 100%[=====================>] 392.08K --.-KB/s in 0.08s 2015-01-03 06:41:10 (4.73 MB/s) - 'openslide-3.4.0.tar.xz.2' saved [401488/40148 8] Example run of the same binary from the Cygwin shell, with %2B properly handled: $ src/wget https://github.com/openslide/openslide/releases/download/v3.4.0/openslide-3.4.0.tar.xz --2015-01-03 06:42:26-- https://github.com/openslide/openslide/releases/download/v3.4.0/openslide-3.4.0.tar.xz Resolving github.com (github.com)... 192.30.252.131 Connecting to github.com (github.com)|192.30.252.131|:443... connected. HTTP request sent, awaiting response... 302 Found Location: https://s3.amazonaws.com/github-cloud/releases/827644/96a6c9f0-8629-11e3-9b05-739b7f3aed83.xz?response-content-disposition=attachment%3B%20filename%3Dopenslide-3.4.0.tar.xz&response-content-type=application/octet-stream&AWSAccessKeyId=AKIAISTNZFOVBIJMK3TQ&Expires=1420267407&Signature=YB9b5SgW7eyUX%2BSWK5L4A0CxzdA%3D [following] --2015-01-03 06:42:27-- https://s3.amazonaws.com/github-cloud/releases/827644/96a6c9f0-8629-11e3-9b05-739b7f3aed83.xz?response-content-disposition=attachment%3B%20filename%3Dopenslide-3.4.0.tar.xz&response-content-type=application/octet-stream&AWSAccessKeyId=AKIAISTNZFOVBIJMK3TQ&Expires=1420267407&Signature=YB9b5SgW7eyUX%2BSWK5L4A0CxzdA%3D Resolving s3.amazonaws.com (s3.amazonaws.com)... 54.231.13.0 Connecting to s3.amazonaws.com (s3.amazonaws.com)|54.231.13.0|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 401488 (392K) [application/octet-stream] Saving to: ‘openslide-3.4.0.tar.xz.9’ openslide-3.4.0.tar 100%[=====================>] 392.08K --.-KB/s in 0.1s 2015-01-03 06:42:27 (3.66 MB/s) - ‘openslide-3.4.0.tar.xz.9’ saved [401488/401488] Thanks, --Benjamin Gilbert
