Re: [Dovecot] Processing too fast?
On Mon, 2007-05-14 at 14:56 -0700, Daniel L. Miller wrote: Is Dovecot grabbing the file from new and moving it to cur faster than ASSP can write? Possible, if it uses inotify there. How can I get around this? ASSP only allows for a folder to save each category to - I can't specify an LDA. Well if it's writing maildir it should open a new file in tmp/, write to it and then move it to new/ instead of opening a file in new/ and writing to it. johannes signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Semi-static userdb...?
On 15.5.2007, at 5.16, John Robinson wrote: One possibility would be to set uid_file=/vmail/%d gid_file=/ vmail/%d. I guess that would be good. Added to TODO, but I'm not sure when I get around to implementing it. Something like the attached? Otherwise it's OK, but I'd want it to work with all userdbs. Looks like the code doesn't currently support doing that in any easy way. With passdbs it'd have been easy to use auth_request_set_field(). I guess I'll add a similar auth_request_add_userdb_field() for CVS HEAD. [1] Use uid_file=/vmail/%d and login with domain ../etc/passwd and you end up looking at /etc/passwd. I don't know whether this matters: it's only doing a stat(), hopefully dovecot won't let you check mail as root, and anyway presumably the user has already had their password checked by now, and someone logging in as [EMAIL PROTECTED]/etc/ passwd or whatever would have failed a password check. Right. Hmm. Although this makes me think about deliver.. If using PAM/ checkpassword and userdb static, Dovecot can't verify that the user exists. Now if the username or domain contains ../ in it, the %n and % d variables in mail location will contain ../ also, which could mean that deliver can write to locations where it's not ever supposed to be writing anything. I think the main problem is that username validations and translations are done only for authentications, not for deliver's userdb lookups. I guess I'll have to change this. After that '/' will be in invalid character list by default. + auth_request_log_info(auth_request, static-userdb, + Can't stat uid_file %s: %s, + value, strerror(errno)); Instead of %s strerror() you can use %m which does the same. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Thinking Outside the Box - Extending IMAP
On Mon, 2007-05-14 at 11:39 -0500, Eric Rostetter wrote: You can setup a ssh tunnel on the server on any port. The user then sets up to connect to that port. The authentication can be done anyway you want, or not at all. We're not talking ssh logins to the server, we're talking ssh tunneling. Actually, I was thinking ssh logins :) What I was thinking is this: * user logs in via ssh * ssh exposes a bunch of services like sftp and doesn't expose shell (iirc this is possible), except the service wouldn't be sftp but 'imap' * the imap service you provide is a pre-authenticated imap session so that authentication/encryption is in ssh. I read my mail this way all the time. * the ssh also provides a few other services that you can use Yes, this'll need a new client, but no, it won't require installing putty, it'll require adding an ssh-transport to the imap protocol in your mail client. Evolution is capable of doing that already by a preauth tunnel command. But you need to modify the client anyway, so this isn't a concern. Thus, what you get is exactly what you want: a service that provides multiple virtual services within a single existing connection. You ought to be able to partition users within ssh, maybe not with openssh but surely with some other implementation, virtual users should be possible too, there's nothing in the ssh protocol that forbids this. Now, at this point, you might want to support imap-only clients. But that's probably easy too, just set up *dovecot* to do the authentication (might sound odd first) and then write a dovecot PAM module that allows ssh to authenticate against dovecot's auth service. Trivial, really. Then you can *also* expose plain imap (with TLS) on another port and therefore support legacy clients. johannes signature.asc Description: This is a digitally signed message part
Re: [Dovecot] Question: contention with blackberry
On 14.5.2007, at 23.09, Stewart Dean wrote: My production imap is currently UWIMAP with mbox.. One of the problems commonly observed is mailbox lock loss, which happens when: a) a VIP has multiple secretaries accessing a single mailbox (actually, they are professional enough to have figured the realities of conflicting access and rarely have a problem), b) somebody leaving their machine on at home and coming in and firing up their work computer c) important people with Blackberriesand the BB service polls the mailbox every so often and breaks the lock. In any case, the maillog ends up with a message like this: May 5 19:45:04 mercury mail:info imapd[2035774]: Logout user=x host=bda056.bis.na.blackberry.com [216.9.249.56] May 5 19:46:52 mercury mail:info imapd[2502902]: Killed (lost mailbox lock) user=x host=cpe-24-161-103-11.hvc.res.rr.com [24.161.103.11] How will this work with Dovecot? None of these are a problem with Dovecot. It supports multiple connections accessing the same mbox. I'm still using mboxes myself (going to move to dbox some day) and I've my desktop and my laptop accessing the same mboxes just fine. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] maildir_file_do(/var/admhome/user/Maildir) racing
On 14.5.2007, at 17.54, Ralf Hildebrandt wrote: One, just one (!) of our users gets: May 14 16:47:17 postamt dovecot: imap-login: Login: user=username, method=PLAIN, rip=141.42.x.y, lip=ip.of.the.server May 14 16:47:17 postamt dovecot: imap-login: Login: user=username, method=PLAIN, rip=141.42.x.y, lip=ip.of.the.server May 14 16:47:17 postamt dovecot: IMAP(username): maildir_file_do(/ var/admhome/username/Maildir) racing This happens at every login? I thought I'd never see this racing error. It means that Dovecot tried to some operation to a maildir file. But the operation failed, because the file was just renamed (eg. flag changed). So it located the file again, and found it. Then it tried the operation again, but it failed the same way. This was tried 10 times until it gave this racing error. I think it's a bit unlikely that the filename was changing that fast. So there's probably a bug somewhere.. Could get strace output from those two logins and send them to me? You can do this with eg.: mail_executable = /usr/local/bin/run-imap.sh run-imap.sh: #!/bin/sh exec strace -tt -s 1000 -o /tmp/imap.$USER.`time +%u`.$$.log /usr/ local/libexec/dovecot/imap PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] maildir_file_do(/var/admhome/user/Maildir) racing
* Timo Sirainen [EMAIL PROTECTED]: This happens at every login? I thought I'd never see this racing error. I found the reason: The User had a symlink in his $HOME/Maildir/cur, which pointed to a NON-existing file :( Courier-IMAP ignored that, dovecot bailed out. I removed the bogus symlink and all is well now. It means that Dovecot tried to some operation to a maildir file. Yep. -- Ralf Hildebrandt (i.A. des IT-Zentrums) [EMAIL PROTECTED] Charite - Universitätsmedizin BerlinTel. +49 (0)30-450 570-155 Gemeinsame Einrichtung von FU- und HU-BerlinFax. +49 (0)30-450 570-962 IT-Zentrum Standort CBFsend no mail to [EMAIL PROTECTED]
Re: [Dovecot] maildir_file_do(/var/admhome/user/Maildir) racing
On Tue, 2007-05-15 at 11:48 +0200, Ralf Hildebrandt wrote: * Timo Sirainen [EMAIL PROTECTED]: This happens at every login? I thought I'd never see this racing error. I found the reason: The User had a symlink in his $HOME/Maildir/cur, which pointed to a NON-existing file :( I've changed Dovecot's behavior now also: x fetch * body imap(tss): Error: Maildir: Symlink destination doesn't exist: /home/tss/Maildir/cur/dgdgd * 40 FETCH (BODY (text plain NIL NIL NIL 7bit 0 0 NIL NIL NIL)) x OK Fetch completed. So it logs an error but still goes ahead without disconnecting the client. signature.asc Description: This is a digitally signed message part
Re: [Dovecot] maildir_file_do(/var/admhome/user/Maildir) racing
* Timo Sirainen [EMAIL PROTECTED]: I've changed Dovecot's behavior now also: x fetch * body imap(tss): Error: Maildir: Symlink destination doesn't exist: /home/tss/Maildir/cur/dgdgd * 40 FETCH (BODY (text plain NIL NIL NIL 7bit 0 0 NIL NIL NIL)) x OK Fetch completed. So it logs an error but still goes ahead without disconnecting the client. Great! Thank you. Success story: We were using courier-imap with 9000 User accounts here at Charite and yesterday I switched to dovecot for imap, imaps and pop3s. The load dropped from 50-60 to 2 (!) during noon. Excellent! -- Ralf Hildebrandt (i.A. des IT-Zentrums) [EMAIL PROTECTED] Charite - Universitätsmedizin BerlinTel. +49 (0)30-450 570-155 Gemeinsame Einrichtung von FU- und HU-BerlinFax. +49 (0)30-450 570-962 IT-Zentrum Standort CBFsend no mail to [EMAIL PROTECTED]
Re: [Dovecot] maildir_file_do(/var/admhome/user/Maildir) racing
Hello Ralf, Ralf Hildebrandt [EMAIL PROTECTED], [20070515 - 12:37:36] * Timo Sirainen [EMAIL PROTECTED]: I've changed Dovecot's behavior now also: x fetch * body imap(tss): Error: Maildir: Symlink destination doesn't exist: /home/tss/Maildir/cur/dgdgd * 40 FETCH (BODY (text plain NIL NIL NIL 7bit 0 0 NIL NIL NIL)) x OK Fetch completed. So it logs an error but still goes ahead without disconnecting the client. Great! Thank you. Success story: We were using courier-imap with 9000 User accounts here at Charite and yesterday I switched to dovecot for imap, imaps and pop3s. The load dropped from 50-60 to 2 (!) during noon. Excellent! Are you willing to elaborate a bit about your setup, hardware, config, etc? This would be good to know what mailsystem is behind that values :-) Thanks Stephan
Re: [Dovecot] maildir_file_do(/var/admhome/user/Maildir) racing
* Stephan Holl [EMAIL PROTECTED]: Are you willing to elaborate a bit about your setup, hardware, config, etc? This would be good to know what mailsystem is behind that values :-) Debian/testing, hand-rolled 2.6.21.1-NOHZ-SMP kernel The users are stored in /etc/passwd (for historical reasons (don't ask)) Dual Xeon 2.8GHz with 3GB RAM 1,8TB mailstore (ext3, external SCSI RAID) -- Ralf Hildebrandt (i.A. des IT-Zentrums) [EMAIL PROTECTED] Charite - Universitätsmedizin BerlinTel. +49 (0)30-450 570-155 Gemeinsame Einrichtung von FU- und HU-BerlinFax. +49 (0)30-450 570-962 IT-Zentrum Standort CBFsend no mail to [EMAIL PROTECTED]
Re: [Dovecot] Refreshing Dovecot
On Mon, 14 May 2007 15:39:45 -0400 Charles Marcus [EMAIL PROTECTED] wrote: If I've made configuration changes, is there a way to refresh...without killing and restartingDovecot? kill -HUP pid =or= killall -HUP dovecot Postfix has a reload option... May I suggest the same for dovecot? The dovecot init script may well have a reload option which HUPs dovecot. -- Marshal Newrock Ideal Solution, LLC - http://www.idealso.com
Re: [Dovecot] Refreshing Dovecot
On Tue, 15 May 2007 10:24:36 -0400 Marshal Newrock [EMAIL PROTECTED] wrote: Postfix has a reload option... May I suggest the same for dovecot? The dovecot init script may well have a reload option which HUPs dovecot. In FreeBSD, all you need to do is add the 'restart' flag to the '/usr/local/etc/rc.d' script. -- Gerard I don't care who does the electing as long as I get to do the nominating Boss Tweed signature.asc Description: PGP signature
Re: [Dovecot] Thinking Outside the Box - Extending IMAP
Quoting Johannes Berg [EMAIL PROTECTED]: On Mon, 2007-05-14 at 11:39 -0500, Eric Rostetter wrote: You can setup a ssh tunnel on the server on any port. The user then sets up to connect to that port. The authentication can be done anyway you want, or not at all. We're not talking ssh logins to the server, we're talking ssh tunneling. Actually, I was thinking ssh logins :) Huh... Not sure why, but... This sounds like it would require both ssh server modifications and e-mail client modifications. As such, you may not get a lot of buy in to your idea. At that point, you're almost half way to creating a new protocol anyway... * the imap service you provide is a pre-authenticated imap session so that authentication/encryption is in ssh. I read my mail this way all the time. * the ssh also provides a few other services that you can use Seems to me that instead of adding plugins to dovecot and the e-mail client, you've added subsystems and plugins to the ssh server and e-mail client. So you've just traded one server/client combination for another. Thus, what you get is exactly what you want: a service that provides multiple virtual services within a single existing connection. But since you've had to modify the client and server, why not just do this with any old client/server protocol? What is so special about ssh in this case? I'd rather just tunnel the imap via ssh, and use the existing ssh tunnel to do pre-auth for other services... Seems more trivial, as we're only modifying the client, not the server... But what do I know/care. I've always been happy with multiple protocols. One reason I like multiple protocols, each with their own server code, is that it scales well. I can put each service on a separate machine if I need to, I can re-prioritize them individually, I can proxy them with ease, etc. When you start jamming lots of protocols into one code base, not only is it harder to audit and debug, it is harder to scale. Yes, you can still scale with load balancers and such, but that introduces additional cost and complexity which isn't needed when the services are isolated. But, I guess not every one needs to scale, and not everyone is on the server end (and yes, things always look different from the client end). johannes -- Eric Rostetter The Department of Physics The University of Texas at Austin Go Longhorns!
[Dovecot] Why is direct invocation of dovecot preferred over inet inclusion?
I'm used to putting things in inetd and, 1) should dovecot crash, I'd think it would get restarted if it was invoked from inet's imap-login 2) it would get started well after the boot ...wouldn't that be preferable. Enlighten me, please -- Stewart Dean, Unix System Admin, Henderson Computer Resources Center of Bard College, Annandale-on-Hudson, New York 12504 [EMAIL PROTECTED] voice: 845-758-7475, fax: 845-758-7035
Re: [Dovecot] Why is direct invocation of dovecot preferred over inet inclusion?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Stewart Dean wrote: I'm used to putting things in inetd and, 1) should dovecot crash, I'd think it would get restarted if it was invoked from inet's imap-login 2) it would get started well after the boot ...wouldn't that be preferable. Enlighten me, please this is actually a basic unix question and is absolutely not related to dovecot. essentially inetd is bad because starting up and closing processes consumes lots of resources. If you only have to handle a very small amount of clients, inetd may be right for you, but as soon as you have to deal with more clients, inetd is simply bad. and about starting ... dovecot like almost any other service can of course also be started automatically without using inetd using the usual init scripts (depending on the unix flavour you are using). - -- Udo Rader BestSolution.at EDV Systemhaus GmbH http://www.bestsolution.at -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Mandriva - http://enigmail.mozdev.org iD8DBQFGSczkuhFd84GLxP8RAnYfAJ0WwPCToP09K0Fd26sDRJH4WaQTNQCeIML0 NW4IJkKsPhEZF7pj1mN2Fmc= =J2F4 -END PGP SIGNATURE-
[Dovecot] Getting rid of the common newbie problems
I think the most common Dovecot newbie problems are: - Not looking at the (correct) logs for errors - Authentication problems - Mail location problems - Mail permission problems Below are some of my ideas how I could stop people from asking these questions with future Dovecot (v1.1+) releases. Suggestions welcome. Distribution people especially should say if they're against some change. * Logging The log file problem is the most annoying one, because a lot of the other problems can be solved once the admin figures out that Dovecot is actually giving useful error messages. Often the admin is only looking at the log file where info messages go (eg mail.log) because Dovecot logs its startup message and login messages there, but not where the errors go (eg mail.err). This has happened even with people who in general are experienced sysadmins. Logging to eg. /var/log/dovecot.log by default would be helpful here, but it's probably better to log via syslog by default. Cyrus logs to local6 by default. Perhaps for Dovecot v1.1 I should make that the default too? Looks like in my Debian system the info messages then go to both /var/log/messages and /var/log/syslog. Error messages only go to /var/log/syslog. So there still isn't by default a single log file where the errors are logged, but it might help a bit. * Authentication Authentication problems can usually be solved by telling the user to set auth_debug_passwords=yes and looking at the logs. I'm not sure if there's anything that can be helped in here. Except the logging message could be updated a bit: Aborted login: user=asdga, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications So the last n failed authentications could be added, where n could also be 0. Another possibility would be to make Dovecot remember if there have been any successful logins (/var/lib/dovecot/success file) and if not, give a bit more helpful error messages: - Client gets: NO Authentication failed. Refer to server log for more information. instead of the normal NO Authentication failed. - Log contains: Aborted login: user=asdga, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured, 1 failed authentications (set auth_debug_passwords=yes to debug the problem) I'm not sure if this is a good idea. * Mail location It seems to be difficult for some people to set mail_location. I don't know if anything can be done before Dovecot v2.0 where I'll split it to multiple settings, such as: driver = maildir root_dir = ~/Maildir index_dir = /var/indexes/%u Another problem that seems to be difficult to understand is why the mail directory contains HOME_DIRECTORY_USED_BUT_NOT_GIVEN_BY_USERDB. The whole userdb concept appears to be weird. This could anyway be fixed by giving an error message earlier and failing the login with internal error. Probably the best place to give the error message would be already in the userdb lookup in dovecot-auth, but that would require that doveoct-auth knows if the home directory is really needed, and to give a useful error message it would also need to tell where it's tried to be used (mail_location, or some namespace's location, or ..). Probably too much trouble to be worth it. So the next best thing is to give the error when it's used: Home directory is used in mail_location, but userdb didn't return a home directory It would be nice if it didn't say userdb, but rather the userdb's name. I guess that would be possible if dovecot-auth told master (or deliver) which userdb was used, but that would normally be just extra overhead. * Mail permissions If mail location is difficult for some, then the concept of UIDs are how they're used in Dovecot is pretty much impossible for some to understand. One of the problems is that there exists dovecot user. So people think that their mails should be owned by the dovecot user. Although I've mentioned in everywhere I can think of that this should not be done, it won't help because either people won't read the pages or even if they do, they somehow still fail to ignore it even though it's written in bold. So unless people (and most importantly, distributions) are against it, I think the dovecot user should be renamed. dovelogin perhaps. In future I might split dovecot-auth even more, and then I would like to create a doveauth user as well. Another possibility would be to drop the dovecot user completely and instead use nobody. That would mean that other nobody processes could kill Dovecot's login processes, but that's pretty much it. Once people understand that dovecot isn't the right user, they hit the next problem: How do I tell Dovecot to run as vmail user? I can paste links to wiki pages or tell them to make userdb return uid=vmail, but that just doesn't seem to get through. There needs to be an easier way, and I think I figured out what it is: Add new mail_uid and mail_gid settings to dovecot.conf. Deprecate user_global_uid/gid in dovecot-ldap.conf and make all the
Re: [Dovecot] Thunderbird or Dovecot bug? (multiple keywords)
On Fri, 2007-05-11 at 11:50 -0700, Troy Engel wrote:
Session:
a) add $Label1
b) add $Label2
c) select INBOX.Drafts
d) select INBOX
e) remove $Label2
f) remove $Label1
Alas, I can't spot where the problem is in the conversation; can I zip
up both logs and send to you? (they're pretty large and contain some
private info)
Looking at the logs, the only difference is that in the working case the
$Label2 is returned before $Label1, and in the broken case it's vice
versa.
You could try if the attached patch helps, but looks like this is a
Thunderbird bug if it relies on the order of the returned keywords.
? src/lib-imap/htpasswd
Index: src/lib-imap/imap-util.c
===
RCS file: /var/lib/cvs/dovecot/src/lib-imap/imap-util.c,v
retrieving revision 1.13
diff -u -r1.13 imap-util.c
--- src/lib-imap/imap-util.c 26 Dec 2004 09:12:40 - 1.13
+++ src/lib-imap/imap-util.c 15 May 2007 17:13:42 -
@@ -5,10 +5,23 @@
#include mail-types.h
#include imap-util.h
+#include stdlib.h
+
+static int neg_strcasecmp_p(const void *p1, const void *p2)
+{
+ const char *const *s1 = p1, *const *s2 = p2;
+ int ret;
+
+ ret = strcasecmp(*s1, *s2);
+ return ret 0 ? 1 :
+ (ret 1 ? -1 : 0);
+}
+
void imap_write_flags(string_t *dest, enum mail_flags flags,
const char *const *keywords)
{
size_t size;
+ const char **keywords2;
size = str_len(dest);
if ((flags MAIL_ANSWERED) != 0)
@@ -25,11 +38,17 @@
str_append(dest, \\Recent );
if (keywords != NULL) {
+ unsigned int len = strarray_length(keywords);
+
+ keywords2 = t_new(const char *, len + 1);
+ memcpy(keywords2, keywords, sizeof(const char *) * len);
+ qsort(keywords2, len, sizeof(const char *), neg_strcasecmp_p);
+
/* we have keywords too */
- while (*keywords != NULL) {
- str_append(dest, *keywords);
+ while (*keywords2 != NULL) {
+ str_append(dest, *keywords2);
str_append_c(dest, ' ');
- keywords++;
+ keywords2++;
}
}
signature.asc
Description: This is a digitally signed message part
Re: [Dovecot] Deliver sending bounces from 'MAILER-DAEMON@'
On Mon, 2007-05-14 at 12:22 +0200, Erland Nylend wrote: Hello Timo, On 2007-05-11, 16:18, Timo Sirainen wrote: On Tue, 2007-04-24 at 12:49 +0200, Erland Nylend wrote: | Apr 24 11:42:08 smtp2 postfix/qmgr[6176]: 05BAE3B67E: from=, size=3055, nrcpt=1 (queue active) | Apr 24 11:42:08 smtp2 postfix/qmgr[6176]: 05BAE3B67E: to=MAILER-DAEMON@, relay=none, delay=0.01, delays=0.01/0.01/0/0, dsn=5.1.3, status=bounced (bad address syntax) .. It seems to me that dovecot is sending bounce messages from 'MAILER-DAEMON@' .. It says to=MAILER-DAEMON@ so I'd guess the Return-Path: header in the message was that? Could you have a look at this case again? The messages to deliver have the smtp mail from and Return-Path set to . (they're bounces) I guess it's Postfix that adds the MAILER-DAEMON there. I get this: May 15 20:22:39 hurina postfix/qmgr[31650]: 00615287D8C86: to=@, relay=none, delay=0.24, delays=0.12/0.13/0/0, dsn=5.1.3, status=bounced (bad address syntax) Anyway, what it was supposed to do was just log it: deliver(tss): May 15 20:24:41 Info: msgid=: Return-Path missing, rejection reason: Quota exceeded Fixed: http://dovecot.org/list/dovecot-cvs/2007-May/008817.html signature.asc Description: This is a digitally signed message part
[Dovecot] deleted Messages don't go into Trash Folder always
Hi all, most of the time deleted messages aren't moved into trash folder but go into nirwana. Sometimes it works correctly. But a restart does not guarantee this function. This effect exists with different imap clients. Can anybody help please. Rainer Sigl -- - Rainer Sigl, | email: [EMAIL PROTECTED] MPI f. extraterrestrische Physik, | phone: +49 (89) 3-3557 Postfach 1312,| fax: +49 (89) 3-3569 D-85741 Garching, Germany | web: http://www.mpe.mpg.de/~sigl/
