On 2002-01-29 09:56 -0500, Alan Eldridge wrote: > In particular, does wget parse and follow links in an HTML document, > when that document is retrieved (using -r) via the FTP protocol? If > not, why not? > > FTP presents an interesting case because because there are two > independent concepts of traversal that can possibly be applied. One is > based entirely upon metadata -- the filesystem content. The other, > applicable (only) when one is retrieving HTML documents via FTP, is > document content based, and is the model used when retrieving via > HTTP. (HTTP presents the filesystem content as document content, if > it presents it at all, so this "split-personality" effect isn't > obvious like it is with FTP.)
I'm inclined to think that recursive retrieval without parsing is a feature. HTML content is normally served over HTTP. If you want to retrieve HTML through FTP, it's likely because you do *not* want to follow the links. If Wget always parsed HTML, even over FTP, it would be impossible to make a complete mirror a tree that has broken href links or hidden files. The former is now all too common, thanks to IIS's case insensitiveness and IE's tolerance of white space and backslashes in URLs. As for the latter, many HTML trees have files that are not linked to (.htaccess files, or files in directories with no index.html and no automatic index). File-level retrieval of HTML trees is not often needed but sometimes essential. -- Andr� Majorel <URL:http://www.teaser.fr/~amajorel/> std::disclaimer ("Not speaking for my employer");
