This is an initial proposal for naming the files and directories
that Wget creates, based on the URLs of the retrieved documents.
At the moment there are many complaints about Wget failing to save
documents which have '?' in their URLs when running under Windows,
for example. In general, the set of illegal characters in
file-names depends on the the operating system and the file-system
in use. Wget can be compiled for different operating systems, but
doesn't know which file-system is being used - you may get the
oddball who wants to save files to a vfat file-system from Linux
for example! Therefore, there should be some way to override or
augment the set of illegal filename characters using a wgetrc
command, for example.
File-names used within the internals of Wget need to be converted
to an external form which deals with illegal characters or illegal
sequences of characters in the file-name. The internal filename
consists of directory separators ('/'), illegal characters, a
nominated 'escape' character and other (legal) characters.
Illegal characters in the internal file-name can be mapped to an
escape sequence in the external file-name, consisting of the escape
character followed by two hex digits (it is assumed that both the
escape character and the hex digits are legal file-name characters
for the operating system and file-system in use!). Escape
characters in the internal file-name can be mapped to an escape
sequence in the same way.
The directory separator character ('/') in the internal file-name
is usually mapped to the directory hierachy on the file-system, but
if the internal file-name contains two or more consecutive
directory separator characters, some of these will need to be
escaped to avoid trying to create directories with null names. (An
alternate solution is to create a directory whose name consists
solely of a single escape character.)
The external file-names are easily reversible back to the internal
form when necessary.
The obvious candidate for the escape character is the '%'
character, although the escape mechanism for file-names is
logically distinct from the escape mechanism for HTTP. The current
version of Wget for Windows remaps all '%' characters to '@', so
perhaps '@' is a better candidate for the escape character for
Windows. (I'm not sure why Wget does this, as '%' seems to be a
legal file-name character for Windows and MS-DOS. Perhaps it is
for usability reasons due to the command shell's variable
interpolation of '%name%' sequences.) The escape character can be
made operating system dependent, and perhaps could be overridden
with a wgetrc command.
That's my initial proposal anyway. I'm not sure about things such
as UTF-8 should be handled, or if that's an issue at all.
