On Tue, 18 Mar 2003, Andreas Aardal Hanssen wrote: > On Mon, 17 Mar 2003, Charlie Brady wrote: > > >We need to decide what we are doing and why we are doing it. > > The first couple of lines of the IMAPdir draft state what we are trying to > do.
That concerns me greatly. I thought that we were here because we wanted a secure, reliable, manageable IMAP daemon. But it seems that this project has taken the turn to define an abstract data structure. > Other than that, the goal of this specification is to find a way to > store multi level IMAP mailboxes on a file system in an unambiguous way. Hasn't Maildir++/Courier already done that (for better or worse)? > Absolutely. Every server is free to choose a delimiter. However, the > IMAPdir specification suggests (perhaps suggests is too loose) the > seperators / or \, and that's no coincidence. > > / and \ are natural delimiters in a folder hierarchy, and everyone's used > to it. They're not natural, they just happen to have been used by a variety of operating systems. > '.' is not a delimiter, it's just a character that is allowed in > any file name. Now most shells allow escaped characters too. I don't think this is relevant. I think we should focus just on the hierarchy separator used by bincimap, and the file storage used by bincimap. > The '.' comes, from what I understand, from NNTP. The news protocol > exports its news groups list using '.' as the delimiter, probably an > artifact from older days when having '/' in a file name was considered a > problem. There is no reason for other protocols not to export '/' or '\' > as the delimiter. The NNTP history is interesting. Other than that, I don't think we gain anything by discussing other protocols. > >and presumably we can also allow a leading (null) separator. > >The next thing we need to do is to map these folders into file system > >Maildirs. Obviously the mapping must be [un]ambiguous. > >My preference is to use the natural mapping > >foo => $prefix/foo > >foo/bar => $prefix/foo/bar > >foo/bar/ => $prefix/foo/bar > >boo/bar/baz => $prefix/foo/bar/baz > > If you read one of my previous mails on this list, you will see that this > approach does not work very well. You need to consider the facts of > UIDVALIDITY with delete and rename (renaming a superior also means that > inferiors should be renamed, taking UIDVALIDITY values into > consideration), deleting a superior does not imply deleting its inferiors > and that this approach would need a recursive descent traversal while the > one suggested by Maildir++ and IMAPdir does not. As I've said in previous emails, the UIDVALIDITY issue isn't insurmountable, and recursive descent traversal isn't difficult. But I doubt that I'll convince you to try using a directory tree, and it isn't something that needs to be urgently resolved - I can move/rename directories as required to fit in. > This approach also fails for mailbox formats other than Maildir or other > directory based formats. You can not create the same structure using, say, > mbox. You can, except that mbox has the natural limitation that it can't have subfolders. > >We start getting ourselves, and probably others, confused if we try to > >allow either . or / to be used as part of a folder name. The only simple > > To this I do not agree. If '.' is not the hierarchy delimiter, then there > is no reason to disallow its use in mailbox names. There is. '.' is defined in Maildir++ to be the standin character for the heirarchy separator. If we need to map back from the file system to IMAP (during LIST), then we need to know whether '.' is literal or is a standin character. Is "my.stuff" a folder called "stuff" within "my", or a folder called "my.stuff"? Well that probably depends on exactly which IMAP daemon is exporting it at this moment. I don't think that is satisfactory. > With '/'. If someone > wants to make a mailbox called > > "Before/after year 2000" > > Then this should naturally be allowed when a '.' is the delimiter. You've confused me. If '/' is the separator, then "Before/after year 2000" is the folder "after year 2000" within the folder "Before" (RFC 2060 tells us this), and will be stored in the file system as "Before.after year 2000" (Maildir++ tells us to do this). If '.' is the hierarchy separator, then "Before/after year 2000" is the folder "Before/after year 2000", and we have no way to represent it in the file system. We can't store it as "Before.after year 2000", because we would then list it as "Before.after year 2000", which would be wrong. > Likewise, > > "2nd. world war" > > should be allowed when '/' is the delimiter. How would you store it in the file system? How would you list that folder? > Not only does this have an > easy logical implementation, but it is makes sense to users too. Most users won't see the strings that are used in the IMAP protocol, and won't see the file system storage. > The only > character that is not allowed is the hierarchy delimiter. We need to disallow both the separator used in the IMAP protocol, and the one used in the file system storage. (This is why I prefer to use / for both). > >To put that another way: > > If we use . as the hierarchy separator, we must declare / as an illegal > > character in folder names. > > If we use / as the hierarchy separator, and we emulate a folder tree > > after Courier, then we must declare . as an illegal character in folder > > names. > > I agree to the first but not the second. What you seem to mix up here is > the mailbox name as presented in IMAP, versus a mailbox name's > representation on the file system. We can choose an arbitrary delimiter in > IMAP and in the file system, independent of eachother. No we can't, because we need to map consistently from one to the other, and back again, and we can't use / in the file system because you don't like the idea of implementing file storage using nested directories. > Typically, the file system representation will be an encoding of the name > as presented in IMAP. No arguments from me there. > The reason for the exception with the single dot at the start of a mailbox > name is to allow Maildir++ users to convert painlessly. If you want the usage to be painless for users, and you don't want to have a conversion process which renames folders, then you will need to use the leading period consistently. Or have the whole prefix configuration, and let the local admin decide whether to rename folders or leave them alone and use a period prefix. Either will be much better than having any ambiguity or special case. > The side effect > for Maildir++ users is that they can not have a root level folder named > "cur", "new" or "tmp" containing mails. They will have to live with this > until the admin converts their depositories to true IMAPdir. Do you have an algorithm for converting to "true IMAPdir"? That might help me understand exactly what you are proposing. > >OTOH, we have a file and directory hierarchy, and we wish to export a > >subset of it by IMAP. There is no requirement that I can see that all > >files and directories can be referred to via an IMAP folder name. Don't > >invent that requirement. > > This you will have to explain to me. The representation of the mailbox > name _must_ map one to one with the name of the mailbox as exported by > this structure. Each IMAP folder name must map unambigously to a maildir directory path, and that directory path must map back to the same IMAP folder name. But not every directory must have a folder name. Consider /tmp/foo, for instanec. If that makes sense to you, then you should then consider ~/foo, then ~/Maildir/foo. None of these needs to have an IMAP folder representation. > In that sense, yes - in IMAPdir you _must_ be able to > refer to each mailbox name's file representation with an IMAP mailbox. That is only possible if you define certain file representations as not part of IMAPdir - which is the same thing that I am saying. > >I personally have a requirement for bincimap to export a set of folders > >currently managed by a maildir patched UW-imapd. I am satisfied that I can > >move and rename directories as I need to when the time comes to convert > >systems. Nevertheless it would be nice to be able to export the current > >folders as-is, with no migration. > > If we are to represent the structure of all formats exactly the way that > individual clients represent them to their users, then we will never get > anywhere. Agreed. > The file system says something about the structure to an IMAPdir compliant > client. There are no IMAPdir clients. > This client must be required to represent this structure exactly > the way IMAPdir defines it, to the client. Why? Why does IMAPdir matter, except in so far as it helps us (you) build an IMAP daemon which works with existing IMAP clients? > How the client then represents the IMAP server's structure to its user it > of utter irrelevance for this discussion. Agreed. > >> So this would be the exception: either one or two dots at the start of an > >> item in IMAPdir represents one dot in the start of the mailbox name: > >> .foo -> .foo > >> ..foo -> .foo > >> How does that sound? > >Bad :-) > > I have a refinement. A single dot at the start of a filename refers to a > single dot at the start of an IMAP mailbox. This conflicts with Courier/Maildir++. > Anywhere else in the file > name, two consecutive dots represent a single dot and a single dot > represents the hierarchy seperator/delimiter. > > .foo -> .foo > ..foo -> ./foo > .foo..bar -> .foo.bar > .foo.bar -> .foo/bar > > So: > > July 2nd.. -> July 2nd. > July.2nd -> July/2nd > > This is completely unambiguous. Perhaps, but I don't think it is compatible with Courier. This one worries me too: ..foo -> ./foo By that you mean "foo" within ".", do you? -- Charlie
