Re: [Bug-wget] wget doesnt work but curl works !
hi Vishwanath, is it possible to use the last released version of wget? You can find it here: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz I have no clue what changes are contained in the so called Red Hat modified version of wget, I highly suggest to use the last upstream version in order to report bug instead of a modified version. What system are you using? Thanks, Giuseppe Vishwanath Reddy Beemidi bvishwana...@gmail.com writes: Hi, I have trouble getting wget to work when downloading a file using http.curl works fine for the same URL. Both the commands are being run from the same server at the command line, OS : RH Linux mdc1pr012 2.6.18-238.9.1.el5 Following are the commands and the debug messages, any insights into what the problem could be are appreciated. [dsop@mdc1pr012]$ wget -d -S http://www.preprod.abc.com/tools/90067660.csv Setting --server-response (serverresponse) to 1 DEBUG output created by Wget 1.11.4 Red Hat modified on linux-gnu. --2011-10-14 18:00:58-- http://www.preprod.abc.com/tools/90067660.csv Resolving www.preprod.abc.com... 184.31.131.61 Caching www.preprod.abc.com = 184.31.131.61 Connecting to www.preprod.abc.com|184.31.131.61|:80... connected. Created socket 3. Releasing 0x07683bc0 (new refcount 1). ---request begin--- GET /tools/90067660.csv HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: www.preprod.abc.com Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 ---response end--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y=[following] Closed fd 3 --2011-10-14 18:00:58-- http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Resolving fd000xnchegrn02... 11.48.43.72 Caching fd000xnchegrn02 = 11.48.43.72 Connecting to fd000xnchegrn02|11.48.43.72|:80... connected. Created socket 3. Releasing 0x076806e0 (new refcount 1). ---request begin--- GET /?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: fd000xnchegrn02 Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 ---response end--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 Closed fd 3 Authorization failed. Following is the curl trace info for the same URL [dsop@mdc1pr012 ~]$ curl --trace-ascii tr.out -o out.dat http://www.preprodmacys.fds.com/tools/90067660.csv % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 155k 100 155k0 0 449k 0 --:--:-- --:--:-- --:--:-- 550k [dsop@mdc1pr012 ~]$ more tr.out == Info: About to connect() to www.preprod.abc.com port 80 == Info: Trying 184.31.131.61... == Info: connected == Info: Connected to www.preprod.abc.com (184.31.131.61) port 80 = Send header, 186 bytes (0xba) : GET /tools/90067660.csv HTTP/1.1 0022: User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 0062: OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 008b: Host: www.preprod.abc.com 00ab: Accept: */* 00b8: = Recv header, 17 bytes (0x11) : HTTP/1.1 200 OK = Recv header, 25 bytes (0x19) : Server: IBM_HTTP_Server = Recv header, 46 bytes (0x2e) : Last-Modified: Fri, 18 Mar 2011 22:42:46 GMT = Recv header, 30 bytes (0x1e) : ETag: b19b9-26f41-7f2b4580 = Recv header, 22 bytes (0x16) : Accept-Ranges: bytes = Recv header, 26 bytes (0x1a) : Content-Type: text/plain = Recv header, 37 bytes (0x25) : Date: Fri, 14 Oct 2011 22:06:15 GMT = Recv header, 24 bytes
Re: [Bug-wget] wget doesnt work but curl works !
Also, I think Tony Lewis's updated NTLM support went in sometime after 1.11.4. (To my knowledge, which may have gaps, the only Redhat-modified wgets that were seriously different from GNU's, was the 1.10.2 series, which included many incomplete/untested bits from 1.11.4, and maybe a couple of their own changes.) -mjc (2011年11月13日 02:15), Giuseppe Scrivano wrote: hi Vishwanath, is it possible to use the last released version of wget? You can find it here: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz I have no clue what changes are contained in the so called Red Hat modified version of wget, I highly suggest to use the last upstream version in order to report bug instead of a modified version. What system are you using? Thanks, Giuseppe Vishwanath Reddy Beemidi bvishwana...@gmail.com writes: Hi, I have trouble getting wget to work when downloading a file using http.curl works fine for the same URL. Both the commands are being run from the same server at the command line, OS : RH Linux mdc1pr012 2.6.18-238.9.1.el5 Following are the commands and the debug messages, any insights into what the problem could be are appreciated. [dsop@mdc1pr012]$ wget -d -S http://www.preprod.abc.com/tools/90067660.csv Setting --server-response (serverresponse) to 1 DEBUG output created by Wget 1.11.4 Red Hat modified on linux-gnu. --2011-10-14 18:00:58-- http://www.preprod.abc.com/tools/90067660.csv Resolving www.preprod.abc.com... 184.31.131.61 Caching www.preprod.abc.com = 184.31.131.61 Connecting to www.preprod.abc.com|184.31.131.61|:80... connected. Created socket 3. Releasing 0x07683bc0 (new refcount 1). ---request begin--- GET /tools/90067660.csv HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: www.preprod.abc.com Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 ---response end--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y=[following] Closed fd 3 --2011-10-14 18:00:58-- http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Resolving fd000xnchegrn02... 11.48.43.72 Caching fd000xnchegrn02 = 11.48.43.72 Connecting to fd000xnchegrn02|11.48.43.72|:80... connected. Created socket 3. Releasing 0x076806e0 (new refcount 1). ---request begin--- GET /?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: fd000xnchegrn02 Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 ---response end--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 Closed fd 3 Authorization failed. Following is the curl trace info for the same URL [dsop@mdc1pr012 ~]$ curl --trace-ascii tr.out -o out.dat http://www.preprodmacys.fds.com/tools/90067660.csv % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 155k 100 155k0 0 449k 0 --:--:-- --:--:-- --:--:-- 550k [dsop@mdc1pr012 ~]$ more tr.out == Info: About to connect() to www.preprod.abc.com port 80 == Info: Trying 184.31.131.61... == Info: connected == Info: Connected to www.preprod.abc.com (184.31.131.61) port 80 = Send header, 186 bytes (0xba) : GET /tools/90067660.csv HTTP/1.1 0022: User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 0062: OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 008b: Host: www.preprod.abc.com 00ab: Accept: */* 00b8: = Recv header, 17 bytes (0x11) : HTTP/1.1 200 OK = Recv header, 25 bytes (0x19) : Server: IBM_HTTP_Server = Recv
Re: [Bug-wget] wget doesnt work but curl works !
AFAIK, my NTLM changes were never integrated with wget because we didn't have a good mechanism to test them against various configurations. At this point, I don't have access to any NTLM server so I can no longer test them at all. Tony -Original Message- From: bug-wget-bounces+wget=exelana@gnu.org [mailto:bug-wget-bounces+wget=exelana@gnu.org] On Behalf Of Micah Cowan Sent: Sunday, November 13, 2011 12:42 PM To: Giuseppe Scrivano Cc: Vishwanath Reddy Beemidi; bug-wget@gnu.org Subject: Re: [Bug-wget] wget doesnt work but curl works ! Also, I think Tony Lewis's updated NTLM support went in sometime after 1.11.4. (To my knowledge, which may have gaps, the only Redhat-modified wgets that were seriously different from GNU's, was the 1.10.2 series, which included many incomplete/untested bits from 1.11.4, and maybe a couple of their own changes.) -mjc (2011年11月13日 02:15), Giuseppe Scrivano wrote: hi Vishwanath, is it possible to use the last released version of wget? You can find it here: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz I have no clue what changes are contained in the so called Red Hat modified version of wget, I highly suggest to use the last upstream version in order to report bug instead of a modified version. What system are you using? Thanks, Giuseppe Vishwanath Reddy Beemidi bvishwana...@gmail.com writes: Hi, I have trouble getting wget to work when downloading a file using http.curl works fine for the same URL. Both the commands are being run from the same server at the command line, OS : RH Linux mdc1pr012 2.6.18-238.9.1.el5 Following are the commands and the debug messages, any insights into what the problem could be are appreciated. [dsop@mdc1pr012]$ wget -d -S http://www.preprod.abc.com/tools/90067660.csv Setting --server-response (serverresponse) to 1 DEBUG output created by Wget 1.11.4 Red Hat modified on linux-gnu. --2011-10-14 18:00:58-- http://www.preprod.abc.com/tools/90067660.csv Resolving www.preprod.abc.com... 184.31.131.61 Caching www.preprod.abc.com = 184.31.131.61 Connecting to www.preprod.abc.com|184.31.131.61|:80... connected. Created socket 3. Releasing 0x07683bc0 (new refcount 1). ---request begin--- GET /tools/90067660.csv HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: www.preprod.abc.com Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 ---response end--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y=[following] Closed fd 3 --2011-10-14 18:00:58-- http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Resolving fd000xnchegrn02... 11.48.43.72 Caching fd000xnchegrn02 = 11.48.43.72 Connecting to fd000xnchegrn02|11.48.43.72|:80... connected. Created socket 3. Releasing 0x076806e0 (new refcount 1). ---request begin--- GET /?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: fd000xnchegrn02 Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 ---response end--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 Closed fd 3 Authorization failed. Following is the curl trace info for the same URL [dsop@mdc1pr012 ~]$ curl --trace-ascii tr.out -o out.dat http://www.preprodmacys.fds.com/tools/90067660.csv % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 155k 100 155k0 0 449k 0 --:--:-- --:--:-- --:--:-- 550k [dsop@mdc1pr012 ~]$ more tr.out == Info: About
Re: [Bug-wget] wget doesnt work but curl works !
(10/14/2011 03:24 PM), Vishwanath Reddy Beemidi wrote: Hi, I have trouble getting wget to work when downloading a file using http.curl works fine for the same URL. It would appear that the site in question requires NTLM authentication, but you haven't supplied a username/password combination. Also, wget 1.11.4 only supports NTLMv1 I believe; if this is a modern NTLM implementation you may need a newer wget version (not sure which version got the newer NTLM support - may not have been until 1.13.x). -mjc
[Bug-wget] wget doesnt work but curl works !
Hi, I have trouble getting wget to work when downloading a file using http.curl works fine for the same URL. Both the commands are being run from the same server at the command line, OS : RH Linux mdc1pr012 2.6.18-238.9.1.el5 Following are the commands and the debug messages, any insights into what the problem could be are appreciated. [dsop@mdc1pr012]$ wget -d -S http://www.preprod.abc.com/tools/90067660.csv Setting --server-response (serverresponse) to 1 DEBUG output created by Wget 1.11.4 Red Hat modified on linux-gnu. --2011-10-14 18:00:58-- http://www.preprod.abc.com/tools/90067660.csv Resolving www.preprod.abc.com... 184.31.131.61 Caching www.preprod.abc.com = 184.31.131.61 Connecting to www.preprod.abc.com|184.31.131.61|:80... connected. Created socket 3. Releasing 0x07683bc0 (new refcount 1). ---request begin--- GET /tools/90067660.csv HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: www.preprod.abc.com Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 ---response end--- HTTP/1.1 302 Found Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Cache-Control: no-cache Pragma: no-cache Content-Type: text/html; charset=utf-8 Connection: close Content-Length: 925 Location: http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y=[following] Closed fd 3 --2011-10-14 18:00:58-- http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= Resolving fd000xnchegrn02... 11.48.43.72 Caching fd000xnchegrn02 = 11.48.43.72 Connecting to fd000xnchegrn02|11.48.43.72|:80... connected. Created socket 3. Releasing 0x076806e0 (new refcount 1). ---request begin--- GET /?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= HTTP/1.0 User-Agent: Wget/1.11.4 Red Hat modified Accept: */* Host: fd000xnchegrn02 Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 ---response end--- HTTP/1.1 401 Unauthorized Cache-Control: no-cache Pragma: no-cache WWW-Authenticate: NTLM WWW-Authenticate: BASIC realm=Federated_Department_Stores Content-Type: text/html; charset=utf-8 Proxy-Connection: close Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ Connection: close Content-Length: 1114 Closed fd 3 Authorization failed. Following is the curl trace info for the same URL [dsop@mdc1pr012 ~]$ curl --trace-ascii tr.out -o out.dat http://www.preprodmacys.fds.com/tools/90067660.csv % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 155k 100 155k0 0 449k 0 --:--:-- --:--:-- --:--:-- 550k [dsop@mdc1pr012 ~]$ more tr.out == Info: About to connect() to www.preprod.abc.com port 80 == Info: Trying 184.31.131.61... == Info: connected == Info: Connected to www.preprod.abc.com (184.31.131.61) port 80 = Send header, 186 bytes (0xba) : GET /tools/90067660.csv HTTP/1.1 0022: User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 0062: OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 008b: Host: www.preprod.abc.com 00ab: Accept: */* 00b8: = Recv header, 17 bytes (0x11) : HTTP/1.1 200 OK = Recv header, 25 bytes (0x19) : Server: IBM_HTTP_Server = Recv header, 46 bytes (0x2e) : Last-Modified: Fri, 18 Mar 2011 22:42:46 GMT = Recv header, 30 bytes (0x1e) : ETag: b19b9-26f41-7f2b4580 = Recv header, 22 bytes (0x16) : Accept-Ranges: bytes = Recv header, 26 bytes (0x1a) : Content-Type: text/plain = Recv header, 37 bytes (0x25) : Date: Fri, 14 Oct 2011 22:06:15 GMT = Recv header, 24 bytes (0x18) : Content-Length: 159553 = Recv header, 24 bytes (0x18) : Connection: Keep-Alive = Recv header, 127 bytes (0x7f) : Set-Cookie: BC_HA_7039BD33D34B38D1_70737B63=4117DB; Domain=.prep 0040: rodmacys.fds.com; expires=Fri, 14-Oct-11 22:36:15 GMT; Path=/ = Recv header, 70 bytes (0x46) : Set-Cookie: shippingCountry=US; path=/; domain=.preprod.abc 0040: .com = Recv header, 104 bytes (0x68) : Set-Cookie: stop_mobi=yes; expires=Fri, 14-Oct-2011 22:36:15 GMT 0040: ; path=/; domain=.preprod.abc.com = Recv data, 5078 bytes (0x13d6)
