RE: wget + dowbloading AV signature files

[Tony Lewis]

Gerard Seibert wrote:

 Is it possible for wget to compare the file named AV.hdb'
 located in one directory, and if it is older than the AV.hdb.gz file
 located on the remote server, to download the AV.hdb.gz file to the
 temporary directory?

No, you can only get wget to compare a file of the same name between your
local system and the remote server.

 The only option I have come up with is to keep a copy of the gz file
 in the temporary directory and run wget from there.

You will need to keep the original gz file with a timestamp matching the
server in order for wget to know that the file you have is the same as the
one on the server.

 Unfortunately, at least as far as I can tell, wget does not issue an
 exit code if it has downloaded a newer file.

Better exit codes is on the wish list.

 It would really be nice though if wget simply issued an exit code if
 an updated file were downloaded.

Yes, it would.

 Therefore, I am unable to craft a script that will unpack the file,
 test and install it if a newer version has been downloaded.

Keep one directory that matches the server and another one (or perhaps two)
where you process new files. Before and after wget runs, you can check the
dates on the directory that matches the server. You only need to process
files that changed.

Hope that helps.

Tony

Differentiated exit statuses [wget + dowbloading AV signature files]

[Micah Cowan]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

(Not Cc'ing Gerard, as I'm not sure he wants to be included in this
tangent.)

Tony Lewis wrote:
 Unfortunately, at least as far as I can tell, wget does not issue an
 exit code if it has downloaded a newer file.
 
 Better exit codes is on the wish list.
 
 It would really be nice though if wget simply issued an exit code if
 an updated file were downloaded.
 
 Yes, it would.

I don't think this is what the exit codes differentiation will handle:
Wget really ought to exit with zero status for all success cases. The
Unix idiom is that 0 is success, anything else is a failure of some
sort; so while it would certainly be handy to differentiate between
various types of success, there isn't really a way to do that appropriately.

The exception is that Unix tools will often issue a non-zero exit status
for I didn't have to do anything; so a -N on a file that didn't have
to be downloaded might result in non-zero, giving that differentiation
we needed. However, an important question would be, what should Wget do
for multi-file downloads, where either multiple files were specified on
the command-line, or it's running in recursive-retrieval mode? Say you
specified three files with -N, one needed downloading, one didn't, and
the other was a 404?

I'm thinking, perhaps more important than different exit codes (which
will still be useful for some circumstances: single-file downloads,
spidering, serious failures, I/O and allocation problems),
strictly-defined program-parseable output formats may be more useful.
Something which, for each file, specifies the results plainly.

This could become even more important when we support multiple
simultaneous download streams: we'll want to output parseable updates on
each chunk of downloaded file we retrieve, so we can communicate to
(say) a GUI wrapper as the different streams are running.

You'd probably want to specifically ask for logging in this format, as
it's liable to be less user-friendly than the current output formats
(who wants to read, during a multi-stream fetch, an endless series of
file `index.html': got bytes 4096-5120 lines?).

- --
Micah J. Cowan
Programmer, musician, typesetting enthusiast, gamer...
http://micah.cowan.name/

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFG9WTL7M8hyUobTrERCNP1AJ9igD3zejm34VBlEIyIdx83Q0V9pgCfd0tW
ax1u6l9uaapCZREZHQljep8=
=08dv
-END PGP SIGNATURE-