RE: -N option
I guess I like Mark's --ignore-length strategy. and it looks like this
could work with a fix to Wget found in this patch:
Index: src/ftp.c
===
RCS file: /pack/anoncvs/wget/src/ftp.c,v
retrieving revision 1.61
diff -u -c -r1.61 ftp.c
*** src/ftp.c 2003/01/11 20:12:35 1.61
--- src/ftp.c 2003/07/30 17:43:04
***
*** 1360,1369
tml++;
#endif
/* Compare file sizes only for servers that tell us correct
! values. Assumme sizes being equal for servers that lie
! about file size. */
cor_val = (con-rs == ST_UNIX || con-rs == ST_WINNT);
! eq_size = cor_val ? (local_size == f-size) : 1 ;
if (f-tstamp = tml eq_size)
{
/* Remote file is older, file sizes can be compared and
--- 1360,1370
tml++;
#endif
/* Compare file sizes only for servers that tell us correct
! values. Assume sizes being equal for servers that lie
! about file size, or if givin the ignore length option */
cor_val = (con-rs == ST_UNIX || con-rs == ST_WINNT);
! eq_size = cor_val
!!opt.ignore_length ? (local_size == f-size) : 1;
if (f-tstamp = tml eq_size)
{
/* Remote file is older, file sizes can be compared and
On Tue, 29 Jul 2003, Post, Mark K wrote:
Other than the --ignore-length option I mentioned previously, no. Sorry.
Mark Post
-Original Message-
From: Preston [mailto:[EMAIL PROTECTED]
Sent: Tuesday, July 29, 2003 7:01 PM
To: [EMAIL PROTECTED]
Subject: Re: -N option
To answer questons asked so far: We are using wget version 1.8.2
I have checked the dates on the local file and the remote file and the
local file date is newer. The reason I thought it was still clobbering
despite the newer date on the local was because of the size difference.
I read that in the online manual here:
http://www.gnu.org/manual/wget/html_chapter/wget_5.html#SEC22
At the bottom it says,
If the local file does not exist, or the sizes of the files do not
match, Wget will download the remote file no matter what the time-stamps
say.
I do want newer files on the remote to replace older files on the local
server. Essentially, I want the newest file to remain on the local.
The problem I am having, however is that if we change/update files on
the local, if they are of a different size, the remote copy is
downloaded and clobbers the local no matter what the dates are. I hope
this is clear, sorry if I have not explained the problem well. Let me
know if you have anymore ideas and if you need me to try again to
explain. Thanks for your help.
Preston
[EMAIL PROTECTED]
--
PINE 4.55 Mailer - www.washington.edu/pine/
source-included, proprietary, gratis, text-based, console email clientIndex: src/ftp.c
===
RCS file: /pack/anoncvs/wget/src/ftp.c,v
retrieving revision 1.61
diff -u -c -r1.61 ftp.c
*** src/ftp.c 2003/01/11 20:12:35 1.61
--- src/ftp.c 2003/07/30 17:43:04
***
*** 1360,1369
tml++;
#endif
/* Compare file sizes only for servers that tell us correct
! values. Assumme sizes being equal for servers that lie
! about file size. */
cor_val = (con-rs == ST_UNIX || con-rs == ST_WINNT);
! eq_size = cor_val ? (local_size == f-size) : 1 ;
if (f-tstamp = tml eq_size)
{
/* Remote file is older, file sizes can be compared and
--- 1360,1370
tml++;
#endif
/* Compare file sizes only for servers that tell us correct
! values. Assume sizes being equal for servers that lie
! about file size, or if givin the ignore length option */
cor_val = (con-rs == ST_UNIX || con-rs == ST_WINNT);
! eq_size = cor_val
!!opt.ignore_length ? (local_size == f-size) : 1;
if (f-tstamp = tml eq_size)
{
/* Remote file is older, file sizes can be compared and
RE: -N option
Other than the --ignore-length option I mentioned previously, no. Sorry. Mark Post -Original Message- From: Preston [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 29, 2003 7:01 PM To: [EMAIL PROTECTED] Subject: Re: -N option Aaron S. Hawley wrote: On Tue, 29 Jul 2003, Post, Mark K wrote: .. So, perhaps you need to modify your work practices rather than diddle with the software. Copy the locally updated files to another location so they're not clobbered when the remote version changes. indeed. consider creating local copies by instead just tracking versions of your image files with RCS if its available for your system (and if you aren't already using it): http://www.gnu.org/software/rcs/ To answer questons asked so far: We are using wget version 1.8.2 I have checked the dates on the local file and the remote file and the local file date is newer. The reason I thought it was still clobbering despite the newer date on the local was because of the size difference. I read that in the online manual here: http://www.gnu.org/manual/wget/html_chapter/wget_5.html#SEC22 At the bottom it says, If the local file does not exist, or the sizes of the files do not match, Wget will download the remote file no matter what the time-stamps say. I do want newer files on the remote to replace older files on the local server. Essentially, I want the newest file to remain on the local. The problem I am having, however is that if we change/update files on the local, if they are of a different size, the remote copy is downloaded and clobbers the local no matter what the dates are. I hope this is clear, sorry if I have not explained the problem well. Let me know if you have anymore ideas and if you need me to try again to explain. Thanks for your help. Preston [EMAIL PROTECTED]
Re: -N option gives proxy error
It's a known problem. Timestamping doesn't work with FTP URLs over proxy because the HEAD request is not honored by the proxy for FTP. Note that your Wget is very old and you should upgrade -- but not because of this, because this problem has remained.
Re: -N option gives proxy error
Sorry that was a typing error, the orginal URL was with a (double) // ie. wget -R *ia64*, *mips* ftp://ftp.iitm.ac.in/debian/main/ and wget -N -R *ia64*, *mips* ftp://ftp.iitm.ac.in/debian/main/ I am attatching the debugged log for the second command that gives the proxy error. I have tried it from the command line with the '-N' and the '--timestamping' option and from the wgetrc file using 'timestamping=on' but the proxy server gives error whenever timestamping is turned on. anselm almeida NIO, Goa On Sat, 02 Feb 2002, you wrote: On 2002-02-09 14:35 +0530, Anselm Almeida wrote: If I use wget -R *ia64*, *mips* ftp:/ftp.iitm.ac.in/debian/main/ I am able to download the files But If I use wget -N -R *ia64*, *mips* ftp:/ftp.iitm.ac.in/debian/main/ I get the following, connecting to darya.nio.org:9000 ... connected Proxy request sent, awaitung response ... 500 Error from proxy 2 Mime-version:1.0 3 Proxy-agent: Netscape-Proxy/3.5 4 Content type:text/html Does it change anything if you replace ftp:/ by ftp:// in the URLs ? -- André Majorel [EMAIL PROTECTED] http://www.teaser.fr/~amajorel/ -- arabhi:anselm~/tmp3$ wget -N --debug -R *ia64*,*mips* ftp://ftp.iitm.ac.in/debian/pool/ DEBUG output created by Wget 1.5.3 on linux-gnu. parseurl (ftp://ftp.iitm.ac.in/debian/pool/;) - host ftp.iitm.ac.in - opath debian/pool/ - dir debian/pool - file - ndir debian/pool parseurl (http://darya.nio.org:9000/;) - host darya.nio.org - port 9000 - opath - dir - file - ndir --15:01:04-- ftp://ftp.iitm.ac.in:21/debian/pool/ = `debian/pool/index.html' Connecting to darya.nio.org:9000... Created fd 3. connected! ---request begin--- HEAD ftp://ftp.iitm.ac.in:21/debian/pool/ HTTP/1.0 User-Agent: Wget/1.5.3 Host: ftp.iitm.ac.in:21 Accept: */* Proxy-Authorization: Basic YW5zZWxtOkJhYmwwMGcxcmw= From: [EMAIL PROTECTED] Accept-Language: en ---request end--- Proxy request sent, awaiting response... HTTP/1.0 500 Error from proxy Mime-version: 1.0 Proxy-agent: Netscape-Proxy/3.5 Content-type: text/html Closing fd 3 15:01:04 ERROR 500: Error from proxy. FINISHED --15:01:04-- Downloaded: 0 bytes in 0 files
