-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Alan Thomas wrote: > Idea for future wget versions: It would be nice if I could > invoke wget programmatically and have options like returning data in > buffers versus files (so data can be searched and/or manipulated in > memory),
This can already be done by using wget's -O switch, which directs the output to a specified file (including standard output). A wrapper program could simply read wget's stdout directly into a buffer. However, - -O is only really useful for single downloads, as there is no delineation between separate files. And, I'll admit that I'm not clear how easy this is to do with "100% Pure Java"; it's quite straightforward on Unix systems in most languages. > or at least getting notification of what files have been > downloaded (progress). ...progress reports are already issued to standard error; parsing this wouldn't be too terribly difficult (though it's not currently guaranteed to be stable across releases). Several programs are already doing this, AIUI. > Then it could be more easily and seamlessly > integrated into other software that needs this capability. I would > especially like to be able to invoke wget from Java code. It sounds to me like you're asking for a library version of Wget. There aren't specific plans to support this at the moment, and I'm not sure how much it'd really buy you: high level programming languages such as Java, Python, Perl, etc, tend to ship with good HTTP and HTML-parsing libraries, in which case rigging your own code to do a good chunk of what Wget does, is probably less work than trying to adapt Wget into library form. I'm not saying I'm ruling it out, but I'd need to hear some good cases for it, in contrast to using what's already available on those platforms. However, some changes are in the works (early early planning stages) for Wget to sport a plugin architecture, and if a bit of glue to call out to higher-level languages is added, plugins written in languages such as Java wouldn't be a big sretch. It may well be that restructuring Wget as a library instead of as a standalone app that runs plugins, may be a better solution; it bears discussion. Also planned is a more flexible output system, allowing for arbitrary formatting of downloaded resources (such as .mht's, or tarballs, or whatever), making delineation in a single output stream possible; also, a metadata system for preserving information about what files have been completely downloaded and which were interrupted, what their original URLs were, etc. All of this, however, is a long way from even really being started, especially given our current developer resources. - -- HTH¸ Micah J. Cowan Programmer, musician, typesetting enthusiast, gamer... http://micah.cowan.name/ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHBRsU7M8hyUobTrERCM5rAJ9dgnkDPZbqQMTL2xfsv25fNiZ8QwCaAwbY AXmKyAsiKIV54fVhzsUzVeU= =oWY6 -----END PGP SIGNATURE-----
