wget-1.11.1 (and 1.10/1.10.1) don't handle the .listing file properly when -c is used. It just appends to that file instead of replacing it which means that wget tries to download each file twice when you run the same command twice. Have a look at this log:
>wget -m -nd -c ftp://ftp.redhat.com/pub/redhat/linux/rawhide/ --2008-04-03 15:30:17-- ftp://ftp.redhat.com/pub/redhat/linux/rawhide/ => `.listing' Resolving ftp.redhat.com... 209.132.176.30 Connecting to ftp.redhat.com|209.132.176.30|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD /pub/redhat/linux/rawhide ... done. ==> PASV ... done. ==> LIST ... done. [ <=> ] 259 --.-K/s in 0s 2008-04-03 15:30:19 (1.66 MB/s) - `.listing' saved [259] Already have correct symlink .message -> README --2008-04-03 15:30:19-- ftp://ftp.redhat.com/pub/redhat/linux/rawhide/README => `README' ==> CWD not required. ==> PASV ... done. ==> RETR README ... done. Length: 404 100%[===================================>] 404 --.-K/s in 0.007s 2008-04-03 15:30:21 (59.4 KB/s) - `README' saved [404] FINISHED --2008-04-03 15:30:21-- Downloaded: 2 files, 663 in 0.007s (95.3 KB/s) >cat .listing drwxr-xr-x 2 ftp ftp 4096 Nov 10 2003 . drwxr-xr-x 8 ftp ftp 4096 May 15 2006 .. lrwxrwxrwx 1 ftp ftp 6 Nov 10 2003 .message -> README -rw-r--r-- 1 ftp ftp 404 Nov 10 2003 README >wget -m -nd -c ftp://ftp.redhat.com/pub/redhat/linux/rawhide/ --2008-04-03 15:30:26-- ftp://ftp.redhat.com/pub/redhat/linux/rawhide/ => `.listing' Resolving ftp.redhat.com... 209.132.176.30 Connecting to ftp.redhat.com|209.132.176.30|:21... connected. Logging in as anonymous ... Logged in! ==> SYST ... done. ==> PWD ... done. ==> TYPE I ... done. ==> CWD /pub/redhat/linux/rawhide ... done. ==> PASV ... done. ==> LIST ... done. 100%[++++++++++++++++++=================>] 518 --.-K/s in 0s 2008-04-03 15:30:28 (2.36 MB/s) - `.listing' saved [518] Already have correct symlink .message -> README Remote file no newer than local file `README' -- not retrieving. Already have correct symlink .message -> README Remote file no newer than local file `README' -- not retrieving. FINISHED --2008-04-03 15:30:28-- Downloaded: 1 files, 518 in 0s (4.73 MB/s) >cat .listing drwxr-xr-x 2 ftp ftp 4096 Nov 10 2003 . drwxr-xr-x 8 ftp ftp 4096 May 15 2006 .. lrwxrwxrwx 1 ftp ftp 6 Nov 10 2003 .message -> README -rw-r--r-- 1 ftp ftp 404 Nov 10 2003 README drwxr-xr-x 2 ftp ftp 4096 Nov 10 2003 . drwxr-xr-x 8 ftp ftp 4096 May 15 2006 .. lrwxrwxrwx 1 ftp ftp 6 Nov 10 2003 .message -> README -rw-r--r-- 1 ftp ftp 404 Nov 10 2003 README This happens only when -c is used. Karsten
