hi Vishwanath, is it possible to use the last released version of wget? You can find it here: ftp://ftp.gnu.org/gnu/wget/wget-1.13.4.tar.xz
I have no clue what changes are contained in the so called "Red Hat modified" version of wget, I highly suggest to use the last upstream version in order to report bug instead of a modified version. What system are you using? Thanks, Giuseppe Vishwanath Reddy Beemidi <bvishwana...@gmail.com> writes: > Hi, > > I have trouble getting wget to work when downloading a file using http.curl > works fine for the same URL. > > Both the commands are being run from the same server at the command line, OS > : RH Linux mdc1pr012 2.6.18-238.9.1.el5 > > Following are the commands and the debug messages, any insights into what > the problem could be are appreciated. > > [dsop@mdc1pr012]$ wget -d -S http://www.preprod.abc.com/tools/90067660.csv > > Setting --server-response (serverresponse) to 1 > > DEBUG output created by Wget 1.11.4 Red Hat modified on linux-gnu. > > > --2011-10-14 18:00:58-- http://www.preprod.abc.com/tools/90067660.csv > > Resolving www.preprod.abc.com... 184.31.131.61 > > Caching www.preprod.abc.com => 184.31.131.61 > > Connecting to www.preprod.abc.com|184.31.131.61|:80... connected. > > Created socket 3. > > Releasing 0x0000000007683bc0 (new refcount 1). > > > ---request begin--- > > GET /tools/90067660.csv HTTP/1.0 > > User-Agent: Wget/1.11.4 Red Hat modified > > Accept: */* > > Host: www.preprod.abc.com > > Connection: Keep-Alive > > > ---request end--- > > HTTP request sent, awaiting response... > > ---response begin--- > > HTTP/1.1 302 Found > > Location: > http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= > > Cache-Control: no-cache > > Pragma: no-cache > > Content-Type: text/html; charset=utf-8 > > Connection: close > > Content-Length: 925 > > > ---response end--- > > > HTTP/1.1 302 Found > > Location: > http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= > > Cache-Control: no-cache > > Pragma: no-cache > > Content-Type: text/html; charset=utf-8 > > Connection: close > > Content-Length: 925 > > Location: > http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y=[following] > > Closed fd 3 > > --2011-10-14 18:00:58-- > http://fd000xnchegrn02/?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= > > Resolving fd000xnchegrn02... 11.48.43.72 > > Caching fd000xnchegrn02 => 11.48.43.72 > > Connecting to fd000xnchegrn02|11.48.43.72|:80... connected. > > Created socket 3. > > Releasing 0x00000000076806e0 (new refcount 1). > > > ---request begin--- > > GET > /?cfru=aHR0cDovL3d3dy5wcmVwcm9kbWFjeXMuZmRzLmNvbS90b29scy85MDA2NzY2MC5jc3Y= > HTTP/1.0 > > User-Agent: Wget/1.11.4 Red Hat modified > > Accept: */* > > Host: fd000xnchegrn02 > > Connection: Keep-Alive > > > ---request end--- > > HTTP request sent, awaiting response... > > ---response begin--- > > HTTP/1.1 401 Unauthorized > > Cache-Control: no-cache > > Pragma: no-cache > > WWW-Authenticate: NTLM > > WWW-Authenticate: BASIC realm="Federated_Department_Stores" > > Content-Type: text/html; charset=utf-8 > > Proxy-Connection: close > > Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ > > Connection: close > > Content-Length: 1114 > > > ---response end--- > > > HTTP/1.1 401 Unauthorized > > Cache-Control: no-cache > > Pragma: no-cache > > WWW-Authenticate: NTLM > > WWW-Authenticate: BASIC realm="Federated_Department_Stores" > > Content-Type: text/html; charset=utf-8 > > Proxy-Connection: close > > Set-Cookie: BCSI-CS-3d1fe99b15515258=2; Path=/ > > Connection: close > > Content-Length: 1114 > > Closed fd 3 > > Authorization failed. > > >>>>>>>>>>>>>>>> Following is the curl trace info for the same URL > > [dsop@mdc1pr012 ~]$ curl --trace-ascii tr.out -o out.dat > http://www.preprodmacys.fds.com/tools/90067660.csv > % Total % Received % Xferd Average Speed Time Time Time > Current > Dload Upload Total Spent Left > Speed > 100 155k 100 155k 0 0 449k 0 --:--:-- --:--:-- --:--:-- > 550k > > [dsop@mdc1pr012 ~]$ more tr.out > > == Info: About to connect() to www.preprod.abc.com port 80 > > == Info: Trying 184.31.131.61... == Info: connected > > == Info: Connected to www.preprod.abc.com (184.31.131.61) port 80 > > => Send header, 186 bytes (0xba) > > 0000: GET /tools/90067660.csv HTTP/1.1 > > 0022: User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 > > 0062: OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5 > > 008b: Host: www.preprod.abc.com > > 00ab: Accept: */* > > 00b8: > > <= Recv header, 17 bytes (0x11) > > 0000: HTTP/1.1 200 OK > > <= Recv header, 25 bytes (0x19) > > 0000: Server: IBM_HTTP_Server > > <= Recv header, 46 bytes (0x2e) > > 0000: Last-Modified: Fri, 18 Mar 2011 22:42:46 GMT > > <= Recv header, 30 bytes (0x1e) > > 0000: ETag: "b19b9-26f41-7f2b4580" > > <= Recv header, 22 bytes (0x16) > > 0000: Accept-Ranges: bytes > > <= Recv header, 26 bytes (0x1a) > > 0000: Content-Type: text/plain > > <= Recv header, 37 bytes (0x25) > > 0000: Date: Fri, 14 Oct 2011 22:06:15 GMT > > <= Recv header, 24 bytes (0x18) > > 0000: Content-Length: 159553 > > <= Recv header, 24 bytes (0x18) > > 0000: Connection: Keep-Alive > > <= Recv header, 127 bytes (0x7f) > > 0000: Set-Cookie: BC_HA_7039BD33D34B38D1_70737B63=4117DB; Domain=.prep > > 0040: rodmacys.fds.com; expires=Fri, 14-Oct-11 22:36:15 GMT; Path=/ > > <= Recv header, 70 bytes (0x46) > > 0000: Set-Cookie: shippingCountry=US; path=/; domain=.preprod.abc > > 0040: .com > > <= Recv header, 104 bytes (0x68) > > 0000: Set-Cookie: stop_mobi=yes; expires=Fri, 14-Oct-2011 22:36:15 GMT > > 0040: ; path=/; domain=.preprod.abc.com > > <= Recv data, 5078 bytes (0x13d6) > > 0000: 90067660,po-xx-xx-xx,popup, > > 001d: 90067660,ex-xx-xx-xx,external sites, > > 0043: 90067660,im-xx-xx-xx,imports, > .........MORE LINES.............. > > <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > > Also when we run wget from another server against the same URL it works > fine...the version on the other server is 1.10.4. So we got that version and > installed it on the server mentioned in the above example and tried but we > still got the same error message. > > Following is the debug log from the successful run on the another server : > scripts@md000ysesx92 /home/scripts(0)$ wget -d -S > http://www.preprod.abc.com/tools/90067660.csv > > Setting --server-response (serverresponse) to 1 > > DEBUG output created by Wget 1.10.2 (Red Hat modified) on linux-gnu. > > > --15:03:42-- http://www.preprod.abc.com/tools/90067660.csv > > => `90067660.csv.3' > > Resolving www.preprod.abc.com... 184.31.131.61 > > Caching www.preprod.abc.com => 184.31.131.61 > > Connecting to www.preprod.abc.com|184.31.131.61|:80... connected. > > Created socket 3. > > Releasing 0x000000000063b390 (new refcount 1). > > > ---request begin--- > > GET /tools/90067660.csv HTTP/1.0 > > User-Agent: Wget/1.10.2 (Red Hat modified) > > Accept: */* > > Host: www.preprod.abc.com > > Connection: Keep-Alive > > > ---request end--- > > HTTP request sent, awaiting response... > > ---response begin--- > > HTTP/1.0 200 OK > > Server: IBM_HTTP_Server > > Last-Modified: Fri, 18 Mar 2011 22:42:46 GMT > > ETag: "b19b9-26f41-7f2b4580" > > Accept-Ranges: bytes > > Content-Type: text/plain > > Content-Length: 159553 > > Date: Fri, 14 Oct 2011 22:03:42 GMT > > Connection: keep-alive > > Set-Cookie: BC_HA_7039BD33D34B38D1_70737B63=41174D; Domain=.preprod.abc.com; > expires=Fri, 14-Oct-11 22:33:42 GMT; Path=/ > > Set-Cookie: shippingCountry=US; path=/; domain=.preprod.abc.com > > Set-Cookie: stop_mobi=yes; expires=Fri, 14-Oct-2011 22:33:42 GMT; path=/; > domain=.preprod.abc.com > > > ---response end--- > > > HTTP/1.0 200 OK > > Server: IBM_HTTP_Server > > Last-Modified: Fri, 18 Mar 2011 22:42:46 GMT > > ETag: "b19b9-26f41-7f2b4580" > > Accept-Ranges: bytes > > Content-Type: text/plain > > Content-Length: 159553 > > Date: Fri, 14 Oct 2011 22:03:42 GMT > > Connection: keep-alive > > Set-Cookie: BC_HA_7039BD33D34B38D1_70737B63=41174D; Domain=. > preprod.abc.com; expires=Fri, 14-Oct-11 22:33:42 GMT; Path=/ > > Set-Cookie: shippingCountry=US; path=/; domain=.preprod.abc.com > > Set-Cookie: stop_mobi=yes; expires=Fri, 14-Oct-2011 22:33:42 GMT; path=/; > domain=.preprod.abc.com > > Registered socket 3 for persistent reuse. > > cdm: 1 2 3 4 5 6 7 8 > > Stored cookie preprod.abc.com -1 (ANY) / <permanent> <insecure> [expiry > 2011-10-14 15:33:42] BC_HA_7039BD33D34B38D1_70737B63 41174D > > cdm: 1 2 3 4 5 6 7 8 > > Stored cookie preprod.abc.com -1 (ANY) / <session> <insecure> [expiry none] > shippingCountry US > > cdm: 1 2 3 4 5 6 7 8 > > Stored cookie preprod.abc.com -1 (ANY) / <permanent> <insecure> [expiry > 2011-10-14 15:33:42] stop_mobi yes > > Length: 159,553 (156K) [text/plain] > > > 100%[=================================================================================================================================================================================================>] > 159,553 105.31K/s > > > 15:03:44 (105.15 KB/s) - `90067660.csv.3' saved [159553/159553] > > > > PS : I changed the domain names mentioned in the logs above > > Thanks, > Vishwanath