Howdy folks,
we're using Dovecot 2.3.19 in combination with vmailmgr as the virtual user
management backend (please don't judge, there are plans to move away from it
for various reasons, but as of today it's there), using the wrapper from
https://wiki.dovecot.org/HowTo/VMailMgr - and we're facing an interesting
problem with colons in home directories. I think this problem is not specific
to vmailmgr but happens in any scenario that includes ":" in directory names.
tl;dr:
- The checkvpw-dovecot-wrapper from the Dovecot wiki converts ":" in directory
names to "::" which works fine with general IMAP use, but at least the
autoexpunge plugin and ManageSieve don't handle this escaping and thus break on
those users.
- When patching the ":"->"::" conversion out of the checkvpw-dovecot-wrapper,
Dovecot itself bails on it, interpreting it as a sub-setting.
- The latter is quite surprising because a HOME/mail_home directory is supposed
to be exactly one directory (AFAIK), with no sub-settings like e.g.
mail_location, so I did not expect it to be "interpreted" in any way.
Short story long:
When you set up a user with a name containing dots, vmailmgr converts them to
colons, pretty much like qmail does (qmail maps the username "xy.z" to a file
named ".qmail-xy:z"; vmailmgr creates the directory "./users/xy:z" for that
username). The wrapper from the dovecot wiki escapes ":" into "::" before
exporting the user's directory as the HOME environment variable:
/* escape possible ':' characters before finally exporting */
escape_colon(buffer, MAXLEN);
setenv("HOME", buffer, 1);
The result looks like that:
[root@stardust ~]# doveadm auth login [email protected] supersecret
passdb: [email protected] auth succeeded
extra fields:
[email protected]
userdb extra fields:
[email protected]
home=/home/coltest/./users/xy::z <---------- notice the double colon here
uid=1009
gid=1010
auth_mech=PLAIN
So far, so good: Working with IMAP generally works fine, but there's a catch:
Not every module seems prepared for this "::" syntax. The first notable
exception is the autoexpunge plugin which simply bails on it:
Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock:
file_create_locked(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock)
failed: safe_mkstemp(/home/coltest/./users/xy::z/dovecot.autoexpunge.lock)
failed: No such file or directory
The second notable exception is ManageSieve which takes the escaped directory
name "as is", leading to a situation where the Maildir is in "xy:z" while the
Sieve stuff is being put in "xy::z" where it effectively is never picked up
during delivery (as the stuff is in the wrong location and the .dovecot.sieve
symlink is - silently - not created):
[coltest@stardust ~]$ ls -la users/*
users/xy:z:
total 28
drwx------. 9 coltest coltest 4096 Jun 16 11:39 .
drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 ..
drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 cur
-rw-------. 1 coltest coltest 376 Jun 16 11:40 dovecot.index.log
-rw-------. 1 coltest coltest 2676 Jun 16 11:39 dovecot.list.index.log
-rw-------. 1 coltest coltest 96 Jun 16 11:39 dovecot.mailbox.log
-rw-------. 1 coltest coltest 51 Jun 16 11:39 dovecot-uidlist
-rw-------. 1 coltest coltest 8 Jun 16 11:39 dovecot-uidvalidity
-r--r--r--. 1 coltest coltest 0 Jun 16 11:39 dovecot-uidvalidity.62aafa70
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Drafts
drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 new
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Sent
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Spam
-rw-------. 1 coltest coltest 28 Jun 16 11:39 subscriptions
drwxr-xr-x. 2 coltest coltest 6 Jun 16 11:39 tmp
drwx------. 5 coltest coltest 108 Jun 16 11:39 .Trash
users/xy::z:
total 0
drwx------. 3 coltest coltest 19 Jun 16 11:40 .
drwxr-xr-x. 4 coltest coltest 31 Jun 16 11:40 ..
drwx------. 3 coltest coltest 33 Jun 16 11:40 sieve
So my initial thought was: Why is the ":" even escaped to a "::" in the first
place? It shouldn't be necessary according to this clean workaround that I
found in a posting from 2009 with Timo advising for this type of handling:
> Hmm. Although now that I looked at your code, you seem to be using only
> HOME. And using ~/ in mail_location should work around any ":" issues.
> So what exactly is the problem you're seeing?
https://www.dovecot.org/list/dovecot/2009-June/040590.html
(In the same thread, Timo _also_ introduced the "::" handling in Dovecot.)
This is exactly what we're doing at delivery time: Setting mail_home to the
real pathname including a _single_ colon (not an escaped double colon) and then
pointing mail_location to "~/":
/usr/libexec/dovecot/dovecot-lda -a '$RECIPIENT' -f '$SENDER' -o
mail_home='$MAILDIR' -o mail_location='maildir:~/'
This works well.
So my assumption was: If the checkvpw-dovecot-wrapper returns the HOME
directory of a virtual user; where's even the need to escape a single colon to
a double colon? So I gave it a try and removed that one line of escaping and
recompiled the wrapper. Result:
[root@stardust ~]# doveadm auth login [email protected] supersecret
passdb: [email protected] auth succeeded
extra fields:
[email protected]
userdb extra fields:
[email protected]
home=/home/coltest/./users/xy:z <---------- single colon here now
uid=1009
gid=1010
auth_mech=PLAIN
Using a real IMAP login, though, this lead to the "interesting" situation that
the login itself worked correctly, but _then_ immediately an "Internal error
occurred":
[root@stardust ~]# nc localhost 143
* OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SPECIAL-USE
LITERAL+ STARTTLS AUTH=PLAIN] Dovecot ready.
0 LOGIN [email protected] supersecret
0 OK [CAPABILITY IMAP4rev1 SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT
SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND
URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED
I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH
LIST-STATUS BINARY MOVE SNIPPET=FUZZY PREVIEW=FUZZY PREVIEW STATUS=SIZE
SAVEDATE SPECIAL-USE LITERAL+ NOTIFY SPECIAL-USE] Logged in
* BYE Internal error occurred. Refer to server log for more information.
This is the according log:
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37
imap-login: Info: Login: user=<[email protected]>, method=PLAIN, rip=::1,
lip=::1, mpid=22889, secured
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37
imap([email protected])<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>: Error:
Namespace '': Unknown setting: z
Jun 16 10:04:37 stardust.uberspace.de dovecot[15268]: Jun 16 10:04:37
imap([email protected])<22889><utbow43hgJoAAAAAAAAAAAAAAAAAAAAB>: Info:
Disconnected: Namespace '': Unknown setting: z in=0 out=428 deleted=0
expunged=0 trashed=0 hdr_count=0 hdr_bytes=0 body_count=0 body_bytes=0
So it looks like Dovecot tries to do some interpretation on the HOME value,
splitting it at the ":" and interpret the "z" as some kind of sub-setting -
which a HOME variable (in contrast to, e.g. mail_location) is not supposed to
have.
Here I'm a bit stuck. My idea is to work around this by eliminating the ":"
from directory names altogether by passing an explicit directory name to
vadduser:
[coltest@stardust ~]$ echo supersecret | vadduser --directory=./users/ab.c ab.c
vadduser: user 'ab.c' successfully added
[coltest@stardust ~]$ ls -l users
total 0
drwx------. 5 coltest coltest 39 Jun 16 10:26 ab.c
drwx------. 5 coltest coltest 39 Jun 16 09:31 xy:z
But still, it would be nicer if Dovecot cleanly supported the ":" in directory
names, as it seems to _want_ to support it (by introducing the "::" escaping)
but then only does it halfway.
Could this be fixed in whatever direction, or is there any other advice how we
should handle this?
All the best,
Jonas
