Re: [Bug-wget] CNET download links not working with WGET

[Jeff Givens]

Micah, thanks for your help.  That was the piece that I was missing.  I 
didn't realize it was re-directing to another site.  I was able to find 
out the other site it was going to, download the executable and then I 
just put in a command to re-name the exe file since it was named the 
URL.  Thanks again for your help.

If you read the most recent output of wget that you gave (after 
quoting the URL), it _does_ treat the string of characters as a whole 
URL. The server redirects it to a shorter URL. If I enter that same 
URL into a browser, it does the same redirection there, and results in 
an HTML page, just like what wget gets. That page seems to have some 
JavaScript or something that initiates a separate download of 
something else; I suppose that something else is what you wanted. As 
you may know, wget doesn't execute JavaScript code from a webpage, so 
you'll need to find the real URL to the thing you wanted to download, 
and feed that to wget.

-mjc

On 06/08/2011 09:38 AM, Jeff Givens wrote
No, it's not working. It downloads part of the URL and creates a file
named 3001-8022_4-10804572.html@spi=077d9109e846975d0db9532bd610588f.1
which is 68 KB. I cannot wget to treat the string of characters as a
whole URL. Please help, I really need to get this script working and the
only place to download this file is from CNET.

So... looks like it works, then. Your command shell isn't complaining
about weird command names, wget is clearly requesting the full and
correct URL, it follows redirections, and saves using the final
redirection URL (the latest sources wouldn't follow that last step -
it'd save using the request URI by default).

If you dislike the filename, then provided you have a recent enough
version of wget you can add the --content-disposition option if the
server provides a rename header ("Content-Disposition"); or else use -E
to have wget force the file name to end in .html

-mjc

(05/26/2011 12:19 PM), Jeff Givens wrote:
Hi, I know this is an older topic but thanks for replying. I forgot to
mention I had already what you listed below and this is the output I
get:

C:\DOWNLOAD>wget 
"http://dw.com.com/redir?edId=3&siteId=4&oId=3000-8022_
4-10804572&ontId=8022_4&spi=077d9109e846975d0db9532bd610588f&lop=link&tag=tdw_dl 



text&ltype=dl_dlnow&pid=11665648&mfgId=6290020&merId=6290020&pguid=HFsQLwoOYJQAA 



BuImQcAAAGm&destUrl=http%3A%2F%2Fdownload.cnet.com%2F3001-8022_4-10804572.html%3 



Fspi%3D077d9109e846975d0db9532bd610588f"
--2011-05-02 12:34:20--
http://dw.com.com/redir?edId=3&siteId=4&oId=3000-8022_4
-10804572&ontId=8022_4&spi=077d9109e846975d0db9532bd610588f&lop=link&tag=tdw_dlt 



ext&ltype=dl_dlnow&pid=11665648&mfgId=6290020&merId=6290020&pguid=HFsQLwoOYJQAAB 



uImQcAAAGm&destUrl=http%3A%2F%2Fdownload.cnet.com%2F3001-8022_4-10804572.html%3F 



spi%3D077d9109e846975d0db9532bd610588f
Resolving dw.com.com... 216.239.113.95
Connecting to dw.com.com|216.239.113.95|:80... connected.
HTTP request sent, awaiting response... 302 Found
Location:
http://download.cnet.com/3001-8022_4-10804572.html?spi=077d9109e846975
d0db9532bd610588f [following]
--2011-05-02 12:34:21--
http://download.cnet.com/3001-8022_4-10804572.html?spi=
077d9109e846975d0db9532bd610588f
Resolving download.cnet.com... 64.30.224.58
Connecting to download.cnet.com|64.30.224.58|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified [text/html]
Saving to:
`3001-8022_4-10804572.html@spi=077d9109e846975d0db9532bd610588f.1'

[<=> ] 69,240 77.3K/s in 0.9s

2011-05-02 12:34:22 (77.3 KB/s) -
`3001-8022_4-10804572.html@spi=077d9109e846975
d0db9532bd610588f.1' saved [69240]


C:\DOWNLOAD>

Thanks for your help.

- Jeff