wget( )
Greetings, I think I have come across a bug with wget( ), although I cannot be 100% certain. Here is a simple instance of it (assuming it is indeed a bug): [results in core dump] wget -r --no-parent -A.ptt --no-directories -nv ftp://ftp.ncbi.nih.gov/genomes/Bacteria/Bacillus_anthracis_A2012/ 22:52:09 URL: ftp://ftp.ncbi.nih.gov/genomes/Bacteria/Bacillus_anthracis_A2012/ [1491] - .listing [1] Segmentation fault (core dumped) [this runs without a problem] wget -r --no-parent --no-directories -nv ftp://ftp.ncbi.nih.gov/genomes /Bacteria/Bacillus_anthracis_A2012/ (The -A.ptt being the difference.) If you look at this directory, you'll notice that there are no subdirectories below Bacillus_anthracis_A2012 and you may question why I am using -r, with --no-parent. The answer is that this is just a boiled-down, repeatable instance that I chose. A similar situation arises wheen using -R as with the -A. [odd] When the parent (ftp://ftp.ncbi.nih.gov/genomes/Bacteria/) is the chosen location, using identical input options (including -A and/or -R), some of the bacterium subfolders (others like Bacillus_anthracis_A2012, but not that one) do not behave in this way - they run without a problem. When I run against all ~150 child subdirectories, about 1/3 of them give the error stated above. With neither -A nor -R used, no core dumps occur across all 150. Here is the version information on the program: AUTHOR Originally written by Hrvoje Niksic [EMAIL PROTECTED] digita.com. COPYRIGHT Copyright (c) 1996, 1997, 1998, 2000, 2001 Free Software Foundation, Inc. Permission is granted to make and distribute verbatim copies of this manual provided the copyright notice and this permission notice are preserved on all copies. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documenta tion License, Version 1.1 or any later version published by the Free Software Foundation; with the Invariant Sec tions being ``GNU General Public License'' and ``GNU Free Documentation License'', with no Front-Cover Texts, and with no Back-Cover Texts. A copy of the license is included in the section entitled ``GNU Free Documentation License''. I appreciate your time. Please let me know if you need any further information. Sincerely, Shane Neph
Re: Logon to a secure site with post-data?
John Clarke [EMAIL PROTECTED] writes: The manual says it's possible, but I can't Logon to a secure site with post-data. I've been trying things like: wget -post-data=username=foopassword=bar https://mysecuresite.net/ authenticate.php It should be --post-data, with two dashes. And you also have to take care to save the cookies using `--save-cookies' and load them using `--load-cookies'. In CVS Wget you can also add `--save-session-cookies' to make sure the session cookies also get saved.
Re: Bug in wget 1.9.1 documentation
Tristan Miller [EMAIL PROTECTED] writes: There appears to be a bug in the documentation (man page, etc.) for wget 1.9.1. I think this is a bug in the man page generation process.
Re: wget( )
[EMAIL PROTECTED] writes: I think I have come across a bug with wget( ), although I cannot be 100% certain. Here is a simple instance of it (assuming it is indeed a bug): [results in core dump] wget -r --no-parent -A.ptt --no-directories -nv ftp://ftp.ncbi.nih.gov/genomes/Bacteria/Bacillus_anthracis_A2012/ 22:52:09 URL: ftp://ftp.ncbi.nih.gov/genomes/Bacteria/Bacillus_anthracis_A2012/ [1491] - .listing [1] Segmentation fault (core dumped) Which version of Wget are you using? Some previous versions of Wget had a bug related to -A and FTP. That bug is fixed in 1.9.1. Here is the version information on the program: AUTHOR Originally written by Hrvoje Niksic [EMAIL PROTECTED] digita.com. COPYRIGHT Copyright (c) 1996, 1997, 1998, 2000, 2001 Free Software Foundation, Inc. There is no version information in this output. You need to send us the output of `wget --version' instead.
May bee report of the bug
Hello, I think wget cannot to store more than one cookie at the time. This is a bug? Installed from wget-cvs_1.9.1-20040319_i386.deb Some log entries following: Best regards, Valdas DEBUG output created by Wget 1.9+cvs-dev on linux-gnu. Created socket 8. Releasing 0x8090868 (new refcount 1). Connected 8 to SSL 0x8090948 ---request begin--- POST /public/login.asp HTTP/1.0 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0) Accept: */* Host: multi. Connection: Keep-Alive Content-Type: application/x-www-form-urlencoded Content-Length: 97 ---request end--- [POST data: MfcISAPICommand=LoginPAGE_ID=283135XRez=YRez] HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Server: Microsoft-IIS/5.0 Date: Mon, 12 Jul 2004 12:43:03 GMT Connection: keep-alive Connection: Keep-Alive Content-Length: 436 Content-Type: text/html Received cookie Nr1: Set-Cookie: ASPSESSIONIDCSSQCSQS=IEMNPBBCDFEKLBANCGOEBLPD; path=/ Cache-control: private ---response end--- 200 OK Registered socket 8 for persistent reuse. Stored to file cookie Nr1: Stored cookie multi. -1 (ANY) / session insecure [expiry none] ASPSESSIONIDCSSQCSQS IEMNPBBCDFEKLBANCGOEBLPD Length: 436 [text/html] 0K 100%7.58 KB/s 15:41:55 (7.58 KB/s) - `./2.htm' saved [436/436] Saved all cookies (No1) Saving cookies to ./cookies.txt. Done saving cookies. DEBUG output created by Wget 1.9+cvs-dev on linux-gnu. Read from file cookie (No1) Stored cookie multi. -1 (ANY) / session insecure [expiry none] ASPSESSIONIDCSSQCSQS IEMNPBBCDFEKLBANCGOEBLPD --15:41:55-- https://multi./SSL/MultiWeb.dll = `./3.htm' Created socket 8. Releasing 0x8092178 (new refcount 1). Connected 8 to SSL 0x8092258 ---request begin--- POST /SSL/MultiWeb.dll HTTP/1.0 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0) Accept: */* Host: multi. Connection: Keep-Alive Sending cookie Nr1 Cookie: ASPSESSIONIDCSSQCSQS=IEMNPBBCDFEKLBANCGOEBLPD Content-Type: application/x-www-form-urlencoded Content-Length: 85 ---request end--- [POST data: MfcISAPICommand=LoginPAGE_ID=283135] HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Server: Microsoft-IIS/5.0 Date: Mon, 12 Jul 2004 12:43:03 GMT Cache-Control: no-cache Content-Type: text/html ATTENTION! Received cookies Nr2 and Nr3: Set-Cookie: ltnid1=1089636115E16734; path=/; domain=.lt; Set-Cookie: ltsid1=000232; path=/; domain=.lt; ---response end--- 200 OK cdm: 1 2 3 4 5 6Cookie coming from multi.lt attempted to set domain to lt ATTENTION! Stored cookie Nr2 But NOT cookie Nr3!: Stored cookie multi. -1 (ANY) / session insecure [expiry none] ltnid1 1089636115E16734 Length: unspecified [text/html] 0K .. 143.00 KB/s Closed 8/SSL 0x8092258 15:41:55 (143.00 KB/s) - `./3.htm' saved [2997] Saving cookies to ./cookies.txt. Done saving cookies. DEBUG output created by Wget 1.9+cvs-dev on linux-gnu. Read from file cookies (Nr1, Nr2 BUT NOT Nr3!) Stored cookie multi. -1 (ANY) / session insecure [expiry none] ltnid1 1089636115E16734 Stored cookie multi. -1 (ANY) / session insecure [expiry none] ASPSESSIONIDCSSQCSQS IEMNPBBCDFEKLBANCGOEBLPD --15:41:55-- http://www1./scripts/Net.dll?Sivu?SID=283135 = `./4.htm' Created socket 8. Releasing 0x8086008 (new refcount 1). ---request begin--- GET /scripts/Net.dll?Sivu?SID=283135 HTTP/1.0 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 4.0) Accept: */* Host: www1. Connection: Keep-Alive ---request end--- HTTP request sent, awaiting response... ---response begin--- HTTP/1.1 200 OK Server: Microsoft-IIS/5.0 Date: Mon, 12 Jul 2004 12:41:54 GMT Cache-Control: no-cache Content-Type: text/html ---response end--- 200 OK Length: unspecified [text/html] 0K 10.71 MB/s Closed fd 8 15:41:55 (10.71 MB/s) - `./4.htm' saved [674] Saving cookies to ./cookies.txt. Done saving cookies. Content of cookies file after all sesions Only cookies Nr1 and Nr2 stored: # HTTP cookie file. # Generated by Wget on 2004-07-12 15:41:55. # Edit at your own risk. multi. FALSE / FALSE 0 ASPSESSIONIDCSSQCSQS IEMNPBBCDFEKLBANCGOEBLPD multi. FALSE / FALSE 0 ltnid1 1089636115E16734
question on wget via http proxy
Hello, I am sitting behind a http proxy and need to access the internet through this channel. In most cases this works fine - but there are certain FTP server sites that I can only access via browser or wget. This also is no problem - as long as I need to retrieve data. Problems come up as soon as I need to upload data - this seems to be possible only via netscape 4. All tools that I used (including gftp, kbear, lftp) do not help out. E.g. using gftp I can access ftp.suse.com - but not these sites. As the browser is rahter unreliable in this respect I would like to use another tool. Problem sites are testcase.boulder.ibm.com ftp.software.ibm.com Since wget is able to obtain directoy listings / retrieve data from there is should be possible to also upload data (the browser is able to as well). What is so special about wget that it is able to perform this task ? If I knew, maybe I could find a solution to this problem. I am running LInux SuSE9.0, kernel 2.4.26, wget-1.8.2-301. I have set env variable http_proxy ftp_proxy which make the connection working fine with wget. Any idea ? Thank you Malte Verschicken Sie romantische, coole und witzige Bilder per SMS! Jetzt neu bei WEB.DE FreeMail: http://freemail.web.de/?mc=021193
Re: question on wget via http proxy
Malte Schünemann wrote: Since wget is able to obtain directoy listings / retrieve data from there is should be possible to also upload data Then it would be wput. :-) What is so special about wget that it is able to perform this task? You can learn a LOT about how wget is communicating with the target site by using the --debug argument. Hope that helps a little. Tony
