Downloading multiple pages
I need some help using an application called interchange on my webserver. I have a url www.somewebsite.com/questions/*.html I want to index all files in that questions directory. I am using the following wget command. CODE wget -r -l2 -A html -X cgi-bin -D www.somewebsite.co.uk/ -P /home/httpd/vhosts/somewebsite.co.uk/catalogs/somewebsite/swish_site/ http://www.somewebsite.co.uk/questions/ This only index the index page of this folder. It wil not follow the links on the page. What would be the appropriate command to use to index all pages from that folder.
Possibly bug
Hi, Have been downloading slackware-11.0-install-dvd.iso, but It seems wget downloaded more then filesize and I found: -445900K .. .. .. .. ..119% 18.53 KB/s in wget-log. Regards, Yuriy Padlyak
Re: Possibly bug
The file was probably being uploaded when you started downloading it, so the HTTP server continued sending data even over the initially reported filesize. Just stop wget, and start it again with option -c to resume download. MT Le mercredi 17 janvier 2007 à 18:16 +0200, Yuriy Padlyak a écrit : Hi, Have been downloading slackware-11.0-install-dvd.iso, but It seems wget downloaded more then filesize and I found: -445900K .. .. .. .. ..119% 18.53 KB/s in wget-log. Regards, Yuriy Padlyak
Problems with progress on large files
Wget aborts on huge files: $ wget ftp://ftp.fsn.hu/testfiles/128T get: progress.c:965: create_image: Assertion `p - bp-buffer = bp-width' failed. Aborted But wget -q ftp://ftp.fsn.hu/testfiles/128T works fine. I tried Wget 1.10+devel from svn trunk branch, Revision 2202. Compiled with gcc 3.4.6 on 32-bit GNU/Linux system with glibc-2.3.6. $ uname -a Linux slack 2.6.19.2 #1 Thu Jan 11 11:40:47 MSK 2007 i686 pentium4 i386 GNU/Linux -- With Best Regards, Stas. All You Need Is Love! Homo sum et nihil humani a me alienum puto.
Addition to problems with progress on large files
I've noticed problem is related with ETA calculating and on systems with high bandwidth it may not fail. It better to test something like $ wget --limit-rate=1024 ftp://ftp.fsn.hu/testfiles/128T -- With Best Regards, Stas. All You Need Is Love! Homo sum et nihil humani a me alienum puto.
Re: Possibly bug
From: Yuriy Padlyak Have been downloading slackware-11.0-install-dvd.iso, but It seems wget downloaded more then filesize and I found: -445900K .. .. .. .. ..119% 18.53 KB/s in wget-log. As usual, it would help if you provided some basic information. Which wget version (wget -V)? On which system type? OS and version? Guesswork follows. Wget versions before 1.10 did not support large files, and a DVD image could easily exceed 2GB. Negative file sizes are a common symptom when using a small-file program with large files. Steven M. Schweda [EMAIL PROTECTED] 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
Re: Downloading multiple pages
From: graham hadgraft I need some help using an application [...] You seem to need some help asking for help. wget -r -l2 -A html -X cgi-bin -D www.somewebsite.co.uk/ -P /home/httpd/vhosts/somewebsite.co.uk/catalogs/somewebsite/swish_site/ http://www.somewebsite.co.uk/questions/ This only index the index page of this folder. It wil not follow the links on the page. What would be the appropriate command to use to index all pages from that folder. Did it occur to you that it might matter which version of wget you're using, and on which system type (and version)? Or that it might be difficult for someone else to guess what happens when no one else can see the Web page which seems to be causing your trouble? Does it actually have links to other pages? Steven M. Schweda [EMAIL PROTECTED] 382 South Warwick Street(+1) 651-699-9818 Saint Paul MN 55105-2547
