Tomas Hjelmberg <[EMAIL PROTECTED]> writes: > Isn't it a good idea to have an option to forbid wget to download files with > the tag: > <meta name="robots" content="noindex">
But how will Wget know not to download those files unless it has already downloaded and inspected them? Or, do you mean that they should be deleted afterwards?
