RE: wget hangs or downloads end up incomplete in Windows 2000 X P.
I tried --passive-ftp parameter on the Windows 2000 machine, and it hung during a file download with 3 seconds left. Never timed out. :( -- Ants. There's a size for every picnic, a colour for every occasion, and a bite to let you know that they are there. Apart from the bite, that could describe a Basenji pretty well. --Len Reddie /\___/\ / /\ /\ \ Phillip Pi (Ant) @ The Ant Farm: http://antfarm.ma.cx | |o o| | E-mail: [EMAIL PROTECTED] or [EMAIL PROTECTED] \ _ /Be sure you removed ANT from e-mail address if you get ( ) a returned e-mail. On Thu, 20 May 2004, Phillip Pi wrote: Not really. I know XP has its ICF enabled but 2000 doesn't have one. I can try it and see what happens. I added --passive-ftp parameter after wget.exe part. I doubt it will makes any differences. On Thu, 20 May 2004, Post, Mark K wrote: Are you behind a firewall or proxy of some kind? If so, you might want to try using passive FTP mode. Mark Post -Original Message- From: Phillip Pi [mailto:[EMAIL PROTECTED] Sent: Thursday, May 20, 2004 3:08 PM To: [EMAIL PROTECTED] Subject: RE: wget hangs or downloads end up incomplete in Windows 2000 X P. FYI. I noticed if I ctrl-c to get out of the hanging part and try to resume, my FTP seems to be broken and hangs. I tried manually with ftp.exe command in command line and it froze with dir command: Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\phillip_piftp 192.168.14.18 Connected to 192.168.14.18. 220 USSM-CPD Microsoft FTP Service (Version 5.0). User (192.168.14.18:(none)): domain\username 331 Password required for domain\username. Password: 230 User domain\username logged in. ftp dir 200 PORT command successful. 150 Opening ASCII mode data connection for /bin/ls. [stuck forever until I ctrl-c to break out of it] I either have to reboot the computer OR wait maybe ten minutes to try again with the FTP connection. On Wed, 12 May 2004, Phillip Pi wrote: On Wed, 12 May 2004, Herold Heiko wrote: OK, I did more tests. I noticed -v is already enabled by default since the you probably have verbose=on in your wgetrc file. Good idea. Should I delete wgetrc? I doubt that will fix my problem since I tried on two different Windows machines. FYI. I don't have wgetrc file anywhere. I only have sample.wgetrc file on the machines I used so it sounds like verbose is enabled by default regardless of wgetrc file. 5250K .. .. .. .. .. The timestamp was from almost an hour ago (I was in a meeting) during the download test. Notice it never timed out to retry or abort! Please What happens if you restart wget again with mirror-like options on the same directory tree ? Does it hang again on the same file ? If yes, what if you Which wget parameter options are they? I never noticed it hangs on the same file for each hang in different tests. Please remember sometimes I have missing files when downloads are complete. It is either hang, finish but incomplete, or perfect. Those are the three results I have seen from many tests. try to download that file only ? If not, could you for any chance run a sniffer on that machine (ethereal is free) ? I do not know how to use this network tool. If you can give me instructions I can try! It would be useful to know if really everything is freezed, or if, for example, for some reason the data is just trickling down at 1byte/minute or something similar (stuck in retrasmission?). I have no idea. Does wget have a bytes/bits per second statistics?
RE: wget hangs or downloads end up incomplete in Windows 2000 X P.
Are you behind a firewall or proxy of some kind? If so, you might want to try using passive FTP mode. Mark Post -Original Message- From: Phillip Pi [mailto:[EMAIL PROTECTED] Sent: Thursday, May 20, 2004 3:08 PM To: [EMAIL PROTECTED] Subject: RE: wget hangs or downloads end up incomplete in Windows 2000 X P. FYI. I noticed if I ctrl-c to get out of the hanging part and try to resume, my FTP seems to be broken and hangs. I tried manually with ftp.exe command in command line and it froze with dir command: Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\phillip_piftp 192.168.14.18 Connected to 192.168.14.18. 220 USSM-CPD Microsoft FTP Service (Version 5.0). User (192.168.14.18:(none)): domain\username 331 Password required for domain\username. Password: 230 User domain\username logged in. ftp dir 200 PORT command successful. 150 Opening ASCII mode data connection for /bin/ls. [stuck forever until I ctrl-c to break out of it] I either have to reboot the computer OR wait maybe ten minutes to try again with the FTP connection. -- This is the ant. Treat it with respect. For it may very well be the next dominant lifeform of our planet. --Empire of the Ants movie /\___/\ / /\ /\ \ Phillip Pi (Ant) @ The Ant Farm: http://antfarm.ma.cx | |o o| | E-mail: [EMAIL PROTECTED] or [EMAIL PROTECTED] \ _ /Be sure you removed ANT from e-mail address if you get ( ) a returned e-mail. On Wed, 12 May 2004, Phillip Pi wrote: On Wed, 12 May 2004, Herold Heiko wrote: OK, I did more tests. I noticed -v is already enabled by default since the you probably have verbose=on in your wgetrc file. Good idea. Should I delete wgetrc? I doubt that will fix my problem since I tried on two different Windows machines. FYI. I don't have wgetrc file anywhere. I only have sample.wgetrc file on the machines I used so it sounds like verbose is enabled by default regardless of wgetrc file. 5250K .. .. .. .. .. The timestamp was from almost an hour ago (I was in a meeting) during the download test. Notice it never timed out to retry or abort! Please What happens if you restart wget again with mirror-like options on the same directory tree ? Does it hang again on the same file ? If yes, what if you Which wget parameter options are they? I never noticed it hangs on the same file for each hang in different tests. Please remember sometimes I have missing files when downloads are complete. It is either hang, finish but incomplete, or perfect. Those are the three results I have seen from many tests. try to download that file only ? If not, could you for any chance run a sniffer on that machine (ethereal is free) ? I do not know how to use this network tool. If you can give me instructions I can try! It would be useful to know if really everything is freezed, or if, for example, for some reason the data is just trickling down at 1byte/minute or something similar (stuck in retrasmission?). I have no idea. Does wget have a bytes/bits per second statistics?
RE: wget hangs or downloads end up incomplete in Windows 2000 X P.
Not really. I know XP has its ICF enabled but 2000 doesn't have one. I can try it and see what happens. I added --passive-ftp parameter after wget.exe part. I doubt it will makes any differences. -- This is the ant. Treat it with respect. For it may very well be the next dominant lifeform of our planet. --Empire of the Ants movie /\___/\ / /\ /\ \ Phillip Pi (Ant) @ The Ant Farm: http://antfarm.ma.cx | |o o| | E-mail: [EMAIL PROTECTED] or [EMAIL PROTECTED] \ _ /Be sure you removed ANT from e-mail address if you get ( ) a returned e-mail. On Thu, 20 May 2004, Post, Mark K wrote: Are you behind a firewall or proxy of some kind? If so, you might want to try using passive FTP mode. Mark Post -Original Message- From: Phillip Pi [mailto:[EMAIL PROTECTED] Sent: Thursday, May 20, 2004 3:08 PM To: [EMAIL PROTECTED] Subject: RE: wget hangs or downloads end up incomplete in Windows 2000 X P. FYI. I noticed if I ctrl-c to get out of the hanging part and try to resume, my FTP seems to be broken and hangs. I tried manually with ftp.exe command in command line and it froze with dir command: Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\Documents and Settings\phillip_piftp 192.168.14.18 Connected to 192.168.14.18. 220 USSM-CPD Microsoft FTP Service (Version 5.0). User (192.168.14.18:(none)): domain\username 331 Password required for domain\username. Password: 230 User domain\username logged in. ftp dir 200 PORT command successful. 150 Opening ASCII mode data connection for /bin/ls. [stuck forever until I ctrl-c to break out of it] I either have to reboot the computer OR wait maybe ten minutes to try again with the FTP connection. On Wed, 12 May 2004, Phillip Pi wrote: On Wed, 12 May 2004, Herold Heiko wrote: OK, I did more tests. I noticed -v is already enabled by default since the you probably have verbose=on in your wgetrc file. Good idea. Should I delete wgetrc? I doubt that will fix my problem since I tried on two different Windows machines. FYI. I don't have wgetrc file anywhere. I only have sample.wgetrc file on the machines I used so it sounds like verbose is enabled by default regardless of wgetrc file. 5250K .. .. .. .. .. The timestamp was from almost an hour ago (I was in a meeting) during the download test. Notice it never timed out to retry or abort! Please What happens if you restart wget again with mirror-like options on the same directory tree ? Does it hang again on the same file ? If yes, what if you Which wget parameter options are they? I never noticed it hangs on the same file for each hang in different tests. Please remember sometimes I have missing files when downloads are complete. It is either hang, finish but incomplete, or perfect. Those are the three results I have seen from many tests. try to download that file only ? If not, could you for any chance run a sniffer on that machine (ethereal is free) ? I do not know how to use this network tool. If you can give me instructions I can try! It would be useful to know if really everything is freezed, or if, for example, for some reason the data is just trickling down at 1byte/minute or something similar (stuck in retrasmission?). I have no idea. Does wget have a bytes/bits per second statistics?
RE: wget hangs or downloads end up incomplete in Windows 2000 X P.
From: Phillip Pi [mailto:[EMAIL PROTECTED] OK, I did more tests. I noticed -v is already enabled by default since the you probably have verbose=on in your wgetrc file. 5250K .. .. .. .. .. The timestamp was from almost an hour ago (I was in a meeting) during the download test. Notice it never timed out to retry or abort! Please What happens if you restart wget again with mirror-like options on the same directory tree ? Does it hang again on the same file ? If yes, what if you try to download that file only ? If not, could you for any chance run a sniffer on that machine (ethereal is free) ? It would be useful to know if really everything is freezed, or if, for example, for some reason the data is just trickling down at 1byte/minute or something similar (stuck in retrasmission?). Heiko Herold -- -- PREVINET S.p.A. www.previnet.it -- Heiko Herold [EMAIL PROTECTED] [EMAIL PROTECTED] -- +39-041-5907073 ph -- +39-041-5907472 fax
RE: wget hangs or downloads end up incomplete in Windows 2000 X P.
Did you ever run the download with -v ? What did the log say when wget seemed to hang or regarding to the missing or corrupt files, or regarding the parsing of the directory index (or whatever it was) linking to those files ? If nothing usefull is logged, try again with -d (but be prepared, a huge amount of information will be logged - better redirect it to a file with -o log.txt or -a log.txt). If still nothing comes up, take a look at the server logs, if you can. Heiko -- -- PREVINET S.p.A. www.previnet.it -- Heiko Herold [EMAIL PROTECTED] [EMAIL PROTECTED] -- +39-041-5907073 ph -- +39-041-5907472 fax -Original Message- From: Phillip Pi [mailto:[EMAIL PROTECTED] Sent: Monday, May 10, 2004 10:40 PM To: [EMAIL PROTECTED] Subject: wget hangs or downloads end up incomplete in Windows 2000 XP. Hello. I downloaded wget v1.9.1 (complete) from http://xoomer.virgilio.it/hherold/. I am having problems in downloading almost a GB of files (over 2600 files and 250 folders). Randomly, the download will just stall completely and never resume OR the download completes, but download is corrupted (sometimes missing files and subfolders). I also had this problem with old v1.8.2. The only way to fix is to set pause one second for each process, but this takes too long (almost an hour) with so many files and folders! Here's what I am using in batch file (download.bat -- changed URLs, account, and passwords for sample): @call wget -c -l0 -r -nH -w0 ftp://domain\username:[EMAIL PROTECTED]/Unreleasedbuilds /blah/blah/1/setups/SUBSETUP/* @call wget -c -l0 -r -nH -w0 ftp://domain\username:[EMAIL PROTECTED]/Unreleasedbuilds /blah/blah/1/setups/SUBSETUP/* I used a Pentium 3 1 Ghz system (512 MB of RAM) with Windows 2000 SP4 (all updates) and a Pentium 4 3 Ghz (HT enabled) with Windows XP Home SP1 (all updates). Each computer is using 100mb connection for network. Doing a copy in Windows' Explorer through network share has NO problems. I assume this method is slower transfer compared to wget command. I'd like to use script so I don't have to do this manually. ;) Thank you in advance. :) -- Ants can carry twenty times their weight, which is useful information if you're moving out and you need help getting a potato chip across town. --Ron Darian /\___/\ / /\ /\ \ Phillip Pi (Ant) @ The Ant Farm: http://antfarm.ma.cx | |o o| | E-mail: [EMAIL PROTECTED] or [EMAIL PROTECTED] \ _ /Be sure you removed ANT from e-mail address if you get ( ) a returned e-mail.
RE: wget hangs or downloads end up incomplete in Windows 2000 X P.
OK, I did more tests. I noticed -v is already enabled by default since the results looked the same when I used -v switch. Next, I tried -d and -o switches. The download got stuck. Here's the end of the log for the last file being downloaded: [snipped -- please note I change some IP addresses, paths, etc. for security reasons; hopefully I didn't mess up with the paths and filenames] Closing fd 1896 Closing fd 1908 226 Transfer complete. 10:59:07 (1.05 MB/s) - `Unreleasedbuilds/blah/blah/1/setups/SUBSETUP/foo-Foobar1_CD_Something/Lakers2/1033dotnetredist.exe' saved [20659224] --10:59:07-- ftp://domain%5username:[EMAIL PROTECTED]/Unreleasedbuilds/blah/blah/1/setups/SUBSETUP/foo-Foobar1_CD_Something/Lakers2/1033dotnetredistSp2.exe = `Unreleasedbuilds/blah/blah/1/setups/SUBSETUP/foo-Foobar1_CD_Something/Lakers2/1033dotnetredistSp2.exe' == CWD not required. == PORT ... Master socket fd 1912 bound. using port 4787. -- PORT 10,160,24,85,18,179 200 PORT command successful. done.== RETR 1033dotnetredistSp2.exe ... -- RETR 1033dotnetredistSp2.exe 150 Opening BINARY mode data connection for 1033dotnetredistSp2.exe(6749072 bytes). done. Created socket fd 1904. Length: 6,749,072 0K .. .. .. .. .. 0%1.58 MB/s 50K .. .. .. .. .. 1%1.58 MB/s 100K .. .. .. .. .. 2%1.58 MB/s 150K .. .. .. .. .. 3%1.53 MB/s 200K .. .. .. .. .. 3%1.06 MB/s 250K .. .. .. .. .. 4%1.04 MB/s 300K .. .. .. .. .. 5%1.53 MB/s 350K .. .. .. .. .. 6%1.06 MB/s 400K .. .. .. .. .. 6%1.53 MB/s 450K .. .. .. .. .. 7%1.58 MB/s 500K .. .. .. .. .. 8%1.58 MB/s 550K .. .. .. .. .. 9% 531.91 KB/s 600K .. .. .. .. .. 9%1.04 MB/s 650K .. .. .. .. .. 10%1.58 MB/s 700K .. .. .. .. .. 11%1.04 MB/s 750K .. .. .. .. .. 12%1.04 MB/s 800K .. .. .. .. .. 12%1.58 MB/s 850K .. .. .. .. .. 13%1.58 MB/s 900K .. .. .. .. .. 14%1.53 MB/s 950K .. .. .. .. .. 15%1.06 MB/s 1000K .. .. .. .. .. 15%1.53 MB/s 1050K .. .. .. .. .. 16%1.58 MB/s 1100K .. .. .. .. .. 17%1.04 MB/s 1150K .. .. .. .. .. 18%1.58 MB/s 1200K .. .. .. .. .. 18%1.58 MB/s 1250K .. .. .. .. .. 19%1.04 MB/s 1300K .. .. .. .. .. 20%1.58 MB/s 1350K .. .. .. .. .. 21%1.53 MB/s 1400K .. .. .. .. .. 22%1.58 MB/s 1450K .. .. .. .. .. 22%1.04 MB/s 1500K .. .. .. .. .. 23%1.58 MB/s 1550K .. .. .. .. .. 24%1.04 MB/s 1600K .. .. .. .. .. 25%1.04 MB/s 1650K .. .. .. .. .. 25%1.58 MB/s 1700K .. .. .. .. .. 26%1.58 MB/s 1750K .. .. .. .. .. 27%1.04 MB/s 1800K .. .. .. .. .. 28%1.58 MB/s 1850K .. .. .. .. .. 28%1.53 MB/s 1900K .. .. .. .. .. 29%1.58 MB/s 1950K .. .. .. .. .. 30%1.04 MB/s 2000K .. .. .. .. .. 31%1.58 MB/s 2050K .. .. .. .. .. 31%1.04 MB/s 2100K .. .. .. .. .. 32%1.58 MB/s 2150K .. .. .. .. .. 33%1.58 MB/s 2200K .. .. .. .. .. 34%1.04 MB/s 2250K .. .. .. .. .. 34%1.53 MB/s 2300K .. .. .. .. .. 35%1.06 MB/s 2350K .. .. .. .. .. 36%
