[Dovecot] Search for (any of) multiple terms slow
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, this may be an obvious logical problem I'm not aware of, which cannot be solved any more efficiently... but maybe it's just a bug or there is potential for optimisation in Dovecot (or Thunderbird?). When searching for multiple terms at once (any of) with Thunderbird/Dovecot (using FTS Squat indexes), it takes much longer (not double the time, or a bit more than that, but really much, much longer) than when I search for only one term. Here are some figures (the IMAP commands are shown as issued by Thunderbird and recorded by Dovecot rawlog). While searching, I can see the 'imap' process using 100% CPU time. Searching for xyz in Archive and subfolders 4 secs 32 select Archive 33 uid SEARCH UNDELETED BODY xyz 34 select Archive.2004 35 UID fetch 1:* (FLAGS) 36 uid SEARCH UNDELETED BODY xyz 37 select Archive.2004.2005 38 UID fetch 1:* (FLAGS) 39 uid SEARCH UNDELETED BODY xyz 40 select Archive.2004.2005.2006 41 UID fetch 1:* (FLAGS) 42 uid SEARCH UNDELETED BODY xyz 43 select Archive.2004.2005.2006.2007 44 UID fetch 1:* (FLAGS) 45 uid SEARCH UNDELETED BODY xyz 46 select Archive.2004.2005.2006.2007.2008 47 UID fetch 1:* (FLAGS) 48 uid SEARCH UNDELETED BODY xyz 49 IDLE Searching for xyz and abc (any of) in Archive and subfolders 123 secs 50 select Archive 51 uid SEARCH UNDELETED (OR BODY xyz BODY abc) 52 select Archive.2004 53 UID fetch 1:* (FLAGS) 54 uid SEARCH UNDELETED (OR BODY xyz BODY abc) 55 select Archive.2004.2005 56 UID fetch 1:* (FLAGS) 57 uid SEARCH UNDELETED (OR BODY xyz BODY abc) 58 select Archive.2004.2005.2006 59 UID fetch 1:* (FLAGS) 60 uid SEARCH UNDELETED (OR BODY xyz BODY abc) 61 select Archive.2004.2005.2006.2007 62 UID fetch 1:* (FLAGS) 63 uid SEARCH UNDELETED (OR BODY xyz BODY abc) 64 select Archive.2004.2005.2006.2007.2008 65 UID fetch 1:* (FLAGS) 66 uid SEARCH UNDELETED (OR BODY xyz BODY abc) 67 IDLE Searching for xyz, abc and def in Archive and subfolders 173 secs 68 select Archive 69 uid SEARCH UNDELETED (OR (OR BODY xyz BODY abc) BODY def) 70 select Archive.2004 71 UID fetch 1:* (FLAGS) 72 uid SEARCH UNDELETED (OR (OR BODY xyz BODY abc) BODY def) 73 select Archive.2004.2005 74 UID fetch 1:* (FLAGS) 75 uid SEARCH UNDELETED (OR (OR BODY xyz BODY abc) BODY def) 76 select Archive.2004.2005.2006 77 UID fetch 1:* (FLAGS) 78 uid SEARCH UNDELETED (OR (OR BODY xyz BODY abc) BODY def) 79 select INBOX 80 UID fetch 1:* (FLAGS) 81 check 82 select Archive.2004.2005.2006.2007 83 UID fetch 1:* (FLAGS) 84 uid SEARCH UNDELETED (OR (OR BODY xyz BODY abc) BODY def) 85 select Archive.2004.2005.2006.2007.2008 86 UID fetch 1:* (FLAGS) 87 uid SEARCH UNDELETED (OR (OR BODY xyz BODY abc) BODY def) 88 IDLE Searching for xyz, abc, def and ghi in Archive and subfolders 217 secs 89 select Archive 90 uid SEARCH UNDELETED (OR (OR (OR BODY xyz BODY abc) BODY def) BODY ghi) 91 select Archive.2004 92 UID fetch 1:* (FLAGS) 93 uid SEARCH UNDELETED (OR (OR (OR BODY xyz BODY abc) BODY def) BODY ghi) 94 select Archive.2004.2005 95 UID fetch 1:* (FLAGS) 96 uid SEARCH UNDELETED (OR (OR (OR BODY xyz BODY abc) BODY def) BODY ghi) 97 select Archive.2004.2005.2006 98 UID fetch 1:* (FLAGS) 99 uid SEARCH UNDELETED (OR (OR (OR BODY xyz BODY abc) BODY def) BODY ghi) 100 select Archive.2004.2005.2006.2007 101 UID fetch 1:* (FLAGS) 102 uid SEARCH UNDELETED (OR (OR (OR BODY xyz BODY abc) BODY def) BODY ghi) 103 select Archive.2004.2005.2006.2007.2008 104 UID fetch 1:* (FLAGS) 105 uid SEARCH UNDELETED (OR (OR (OR BODY xyz BODY abc) BODY def) BODY ghi) 106 IDLE Patrick. - -- STAR Software (Shanghai) Co., Ltd.http://www.star-group.net/ Phone:+86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779 PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.7 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org iD8DBQFIooxq7yMg/OiDoAURAhb9AJ4jOaK+eHXcSwJTAER8xEKIlJJt0gCgtZan wJm05V0F03gj+6YxbNOtCew= =jID/ -END PGP SIGNATURE-
Re: [Dovecot] sieve - Sendmail process terminated abnormally, exit status 70
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 12 Aug 2008, Thomas Harold wrote: Check out /usr/include/sysexits.h what exit code 70 means on your system - 70 is internal software error in Linux. Then check when /usr/lib/sendmail will exit with this code. Deliver will run /usr/lib/sendmail with the uid of the target mailbox, you said virtual user - so you've configured the id in dovecot.conf, I guess. E.g. test with: su vuser -c '/usr/lib/sendmail -i -f sender -- target-address' \ testmail ; echo $? Bye, - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) iD8DBQFIopCUVJMDrex4hCIRAohdAJ9qIoQx+XEHbz1vRvqCZP0bDzm2EQCfW61C wvN3iYpFkPPk/v+xRfx0Mjk= =N4J2 -END PGP SIGNATURE-
[Dovecot] RfE: use HOSTNAME environment variable in hostpid_init()
Hi, All parts of dovecot except deliver uses the result of hostpid_init() in src/lib/hostpid.c as a hostname which only asks gethostname(). deliver honours environment variable HOSTNAME in src/deliver/deliver.c: getenv(HOSTNAME); and uses the hostname of hostpid_init() as a fallback. Wouldn't it be consequent to evaluate the environment variable HOSTNAME in hostpid_init() with fallback to gethostname() and remove the special behavior from deliver? The dovecot master process should export $HOSTNAME to all childs in this case. Motivation: Dovecot uses the hostname as part of Maildir file names and gethostname() is not fully qualified on all operating systems (it is on IRIX but not on Linux). But unqualified hostnames are not unique and it is frequent that dovecot hosts are named mail oder imap as hostname. To distinguish between mail.domain1.com and mail.domain2.net would be easier if dovecot evaluate $HOSTNAME in all case not just deliver as program imap can create Maildir files as well. Heiko Heiko Schlichting Freie Universität Berlin [EMAIL PROTECTED] Zentraleinrichtung für Datenverarbeitung (ZEDAT) Telefon +49 30 838-54327 Fabeckstraße 32 Telefax +49 30 838454327 14195 Berlin
Re: [Dovecot] syslog with PID
Timo Sirainen wrote: On Jul 31, 2008, at 12:20 PM, Heiko Schlichting wrote: Without logging the pid, it is impossible to match 'Disconnected' log entries and the corresponding session start/login. You can use %p in mail_log_prefix to log imap/pop3 process PID. It can't be done with login processes, but even if it could be done I doubt it'd be useful in figuring out the matching login and disconnect? Unlike other IMAP servers Dovecot's login process doesn't directly execv() the imap process.. Using mail_log_prefix = %Ls[%p]: user=%u, is ok but it does not fully solve my problem. I've thought about this before too though. Maybe some kind of a unique cookie could be useful. Sounds good but complicated. The only reason why I try to match login and disconnect is that imap and pop3 does not log a starting message (without enabling debug) and login was the only entry during connect. The following minimal patch solves this in a sufficient way and I suggest to adopt it into the official release: --- ./src/imap/main.c.org 2008-07-20 19:57:07.0 +0200 +++ ./src/imap/main.c 2008-08-06 12:23:16.0 +0200 @@ -286,6 +286,7 @@ lib_init(); drop_privileges(); + i_info(Connected); process_title_init(argv, envp); ioloop = io_loop_create(); --- ./src/pop3/main.c.org 2008-07-20 19:57:16.0 +0200 +++ ./src/pop3/main.c 2008-08-06 12:29:11.0 +0200 @@ -268,6 +268,7 @@ lib_init(); drop_privileges(); + i_info(Connected); process_title_init(argv, envp); ioloop = io_loop_create(); Heiko Schlichting Freie Universität Berlin [EMAIL PROTECTED] Zentraleinrichtung für Datenverarbeitung (ZEDAT) Telefon +49 30 838-54327 Fabeckstraße 32 Telefax +49 30 838454327 14195 Berlin pgpKK2dm5kjLk.pgp Description: PGP signature
[Dovecot] Trying to Hide Namespace - new configuration options?
Using 1.1.2
We have:
# default namespace
namespace private {
separator = /
prefix =
inbox = yes
#location = maildir:/home/virtual/%h
#Maildir
}
# for backwards compatibility:
namespace private {
separator = /
prefix = mail/
location = maildir:%h/Maildir/
#Maildir
hidden = yes
list = no
}
The 'hidden=yes' seemed to do nothing but saw earlier in this list a
reply to add 'list=no'. This has worked to hide all folder but we still
have /mail/Trash showing and it refuses to go away even after unsubscribing.
mail/Trash appears when I set my email client to use folder prefix mail/.
If i set the prefix to blank I only see my Inbox and Deleted folders.
Interestingly it shows 'Deleted' not 'Trash'. I guess Thunderbird
renames 'Trash'.
What other configs are there to try? Is there a list of config changes
for 1.1.x anywhere? We were not aware of the list=no config.
Thanks,
Daniel
Re: [Dovecot] How to do line breaks in sieve scripts?
CJ, Sephan, On Friday 08 August 2008 16:37:11 CJ Keist wrote: Thank you, didn't realize how buggy the libphp-sieve was. It does compile fine with sievec. Fixed the issue last night. If you find other bugs, please let me know through the bug-tracker in the future. I rely on feedback to get the last ones out. So, if anyone is fancy to check your script and run into something, let me know. Stephan Bosch wrote: Actually, the libsieve-php is wrong. From RFC 5228 Section 8.1 (http://tools.ietf.org/html/rfc5228): quoted-safe= CRLF / octet-not-qspecial ; either a CRLF pair, OR a single octet other ; than NUL, CR, LF, double-quote, or backslash quoted-text= *(quoted-safe / quoted-special / quoted-other) quoted-string = DQUOTE quoted-text DQUOTE So, presuming that the lines end in CRLF and not just LF (on this part the RFC is too strict in my opinion), your script is valid (I verified). The line breaks were not the problem. Instead I used the wrong pattern modifier so that multiline quoted strings never matched. Classic one-liner fix. Regards Heiko
Re: [Dovecot] Trying to Hide Namespace - new configuration options?
On 8/13/2008, Daniel Watts ([EMAIL PROTECTED]) wrote: If i set the prefix to blank I only see my Inbox and Deleted folders. Interestingly it shows 'Deleted' not 'Trash'. I guess Thunderbird renames 'Trash'. Maybe this is a TBird issue rather than dovecot? The Trash is a 'special folder' to most clients, so it may be trying to force it somehow. -- Best regards, Charles
Re: [Dovecot] Cyrus vs Dovecot
Hello: I think you are missing a point which is most important, i.e., what type of support Cyrus vs Dovecot offers. In my experience: Cyrus = 0 Dovecot= 100 My personal experience. Kirt
Re: [Dovecot] Trying to Hide Namespace - new configuration options?
Charles Marcus wrote: On 8/13/2008, Daniel Watts ([EMAIL PROTECTED]) wrote: If i set the prefix to blank I only see my Inbox and Deleted folders. Interestingly it shows 'Deleted' not 'Trash'. I guess Thunderbird renames 'Trash'. Maybe this is a TBird issue rather than dovecot? The Trash is a 'special folder' to most clients, so it may be trying to force it somehow. Looks like the issue was that before the list=no option was introduced the server had actually created a file on the server. After dovecot was fixed it could read this file but not delete / unsubscribe it. After manually removing all the bad entries on the server it seems to have worked. Still - is there an written update somewhere with the new configs? Thanks, Daniel
Re: [Dovecot] Cyrus vs Dovecot
kbajwa wrote: Hello: I think you are missing a point which is most important, i.e., what type of support Cyrus vs Dovecot offers. In my experience: Cyrus = 0 Dovecot= 100 My personal experience. Don't forget that although for some reason he doesn't widely advertise it - Timo is also for hire if you have special requirements or need a bug fixing even faster. He is very good value and I am not sure why he doesn't say more about his services more publicly... If you had a requirement for a custom feature then please do approach him with your cheque book open and request a feature - I think you will have a very good experience Commercial support can be a real boon for some companies so if this is you then be sure to ask Ed W
Re: [Dovecot] Cyrus vs Dovecot
kbajwa a écrit : Hello: I think you are missing a point which is most important, i.e., what type of support Cyrus vs Dovecot offers. In my experience: Cyrus = 0 Dovecot= 100 My personal experience. Kirt I guess you've right but I can't post this answer at Cyrus mailing list. I'm just trying to have my own opinion of imap server and I already have sarcastic answer on the cyrus mailing list ! regards begin:vcard fn:Mathieu Kretchner n:Kretchner;Mathieu org:INRIA;Syslog adr;dom:;;2007 route des lucioles - BP93;Sophia Antipolis;;06902 CEDEX email;internet:[EMAIL PROTECTED] tel;work:04 92 38 76 67 x-mozilla-html:FALSE version:2.1 end:vcard
Re: [Dovecot] sieve - Sendmail process terminated abnormally, exit status 70
Steffen Kaiser, 13.08.2008 (d.m.y): On Tue, 12 Aug 2008, Thomas Harold wrote: Check out /usr/include/sysexits.h what exit code 70 means on your system - 70 is internal software error in Linux. Then check when /usr/lib/sendmail will exit with this code. Deliver will run /usr/lib/sendmail with the uid of the target mailbox, you said virtual user - so you've configured the id in dovecot.conf, I guess. I just had a similar problem caused by the fact that /usr/lib/sendmail was missing. As I'm using exim as MTA, I created /usr/lib/sendmail as a symlink pointing to the exim binary. Regards, Christian -- Q: What's a WASP's idea of open-mindedness? A: Dating a Canadian.
[Dovecot] Marking as Read causes Body.Peek on ALL messages - Reasonable?
I've noticed when I select a folder of messages (Thunderbird) and mark
them all as read (or unread) it produces the following IMAP transcript:
Source Destination Info
c.c.c.c s.s.s.s Request: DONE
s.s.s.s c.c.c.c Response: 20 OK Idle completed.
c.c.c.c s.s.s.s Request: 21 uid store 1:20 -Flags (\Seen)
s.s.s.s c.c.c.c Response: * 1 FETCH (UID 1 FLAGS (\Recent NonJunk))
c.c.c.c s.s.s.s Request: 22 UID fetch 2 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 2 FETCH (UID 2 RFC822.SIZE 1339 BODY[] {1339}
c.c.c.c s.s.s.s Request: 23 UID fetch 3 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 3 FETCH (UID 3 RFC822.SIZE 1342 BODY[] {1342}
c.c.c.c s.s.s.s Request: 24 UID fetch 5 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 5 FETCH (UID 5 RFC822.SIZE 1344 BODY[] {1344}
c.c.c.c s.s.s.s Request: 25 UID fetch 6 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 6 FETCH (UID 6 RFC822.SIZE 1350 BODY[] {1350}
c.c.c.c s.s.s.s Request: 26 UID fetch 7 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 7 FETCH (UID 7 RFC822.SIZE 1344 BODY[] {1344}
c.c.c.c s.s.s.s Request: 27 UID fetch 8 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 8 FETCH (UID 8 RFC822.SIZE 1325 BODY[] {1325}
c.c.c.c s.s.s.s Request: 28 UID fetch 9 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 9 FETCH (UID 9 RFC822.SIZE 1323 BODY[] {1323}
c.c.c.c s.s.s.s Request: 29 UID fetch 10 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 10 FETCH (UID 10 RFC822.SIZE 1338 BODY[] {1338}
c.c.c.c s.s.s.s Request: 30 UID fetch 11 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 11 FETCH (UID 11 RFC822.SIZE 1333 BODY[] {1333}
c.c.c.c s.s.s.s Request: 31 UID fetch 12 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 12 FETCH (UID 12 RFC822.SIZE 1344 BODY[] {1344}
c.c.c.c s.s.s.s Request: 32 UID fetch 13 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 13 FETCH (UID 13 RFC822.SIZE 1335 BODY[] {1335}
c.c.c.c s.s.s.s Request: 33 UID fetch 15 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 15 FETCH (UID 15 RFC822.SIZE 1323 BODY[] {1323}
c.c.c.c s.s.s.s Request: 34 UID fetch 16 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 16 FETCH (UID 16 RFC822.SIZE 1325 BODY[] {1325}
c.c.c.c s.s.s.s Request: 35 UID fetch 17 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 17 FETCH (UID 17 RFC822.SIZE 1333 BODY[] {1333}
c.c.c.c s.s.s.s Request: 36 UID fetch 18 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 18 FETCH (UID 18 RFC822.SIZE 1327 BODY[] {1327}
c.c.c.c s.s.s.s Request: 37 UID fetch 19 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 19 FETCH (UID 19 RFC822.SIZE 1343 BODY[] {1343}
c.c.c.c s.s.s.s Request: 38 UID fetch 20 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 20 FETCH (UID 20 RFC822.SIZE 1337 BODY[] {1337}
c.c.c.c s.s.s.s Request: 39 IDLE
s.s.s.s c.c.c.c Response: + idling
NOTE:
c.c.c.c = client (thunderbird)
s.s.s.s = server (dovecot 1.1.2)
Is this reasonable? I thought it would be possible to just send a list
of message UIDs and say mark these as /seen /unseen etc.
Of course this could (is probably) just Thunderbird being braindead but
confirmation would be appreciated.
Thanks,
Daniel
Re: [Dovecot] Cyrus vs Dovecot
Mathieu Kretchner wrote: kbajwa a écrit : Hello: I think you are missing a point which is most important, i.e., what type of support Cyrus vs Dovecot offers. In my experience: Cyrus = 0 Dovecot= 100 My personal experience. Kirt I guess you've right but I can't post this answer at Cyrus mailing list. I'm just trying to have my own opinion of imap server and I already have sarcastic answer on the cyrus mailing list ! Reading the cyrus list I think the above quote might be a bit unfair and accidently crossposted? In any case I only have experience of dovecot and it's used in some larger installs such as the old webmail.us, now 11 (I believe). I think your installation is probably large enough that you might want to do a trial migration of a couple of accounts and see if migration is a problem. Certainly for all new servers I would STRONGLY recommend some sort of virtualisation option (I use linux vservers, lots of other options available). This makes it fantasically easy to boot up (say) three instances of your target software installation, perhaps all with different configuration options and compare them easily. I used this as a solution to migrate from Courier and also recently when I was migrating from 32bit to 64bit guests - essentially you spin up your new guest, get it all ready, test it like made and then in a couple of seconds you can down the live guest and boot up the new guest. I separate out all signficant data from the guest partition so try to keep the actual installations under a couple hundred MB each (even that feels bloated, but hey) and this makes it simple to boot up a copy of a guest to test some change without having to copy too much I personally picked dovecot because I worried about the horror stories I read about with cyrus. However, both are clearly the two best options available for opensource solutions right now and both are used in large installations so you should be very happy with either. With regards to functionality it would appear (I don't use cyrus) that cyrus has more admin tools to do stuff, but Dovecot is built to be more hackable, for example you can easily run a script before each (imap, etc) login and hence do some very advanced stuff through that route. Plugins also appear to be quite easy to write to extend dovecot in new directions On the cyrus list they mentioned email retention policies. Now some people are going to say that this is really a job for the MTA (postfix/sendmail/etc). However, you have some plugins which might get you partly towards solving that need, but nothing out of the box which would give you a cast iron (stand up in court) kind of archiving control. However, you can get close I think Ed W
Re: [Dovecot] Cyrus vs Dovecot
On Aug 13, 2008, at 12:44 PM, Ed W wrote: On the cyrus list they mentioned email retention policies. Now some people are going to say that this is really a job for the MTA (postfix/sendmail/etc). However, you have some plugins which might get you partly towards solving that need, but nothing out of the box which would give you a cast iron (stand up in court) kind of archiving control. However, you can get close I think I noticed someone mentioned delayed expunges, which is implemented nearly identically for Dovecot using lazy-expunge plugin. Although with maildir that requires renaming the files elsewhere while Cyrus is able to just leave the files where they are. I'll probably implement the same thing for dbox format some day. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] IMAP inbox: cannot delete tagged messages or change their tag
On Aug 12, 2008, at 7:17 PM, Epimedia wrote: Hello, a bit of a problem with Dovecot. Version is 1.0.15 and is a default install (no changes to config) from FC8 repository... except that Dovecot is configured to use Maildir I have a user on the system who uses the tags in Thunderbird (Important, Personal etc). When she tags the messages they cannot be deleted and their tags cannot be changed. When deleting a message from the IMAP inbox it momentarily disappears and moves a copy to the Trash but when the IMAP inbox is reloaded the message reappears. The same happens when I try to change a tag: it comes back with the previous tag. Untagged messages can be freely deleted. Also, this problem does not happen if the messages are in a IMAP folder. They can be freely deleted. The problem seems specific to Dovecot, not Thunderbird, as I am able to tag and delete messages in another IMAP account. Also, accessing the same IMAP account using OS X Mail the messages come back after deleting them. Where should I start looking for a solution to this problem? Sounds pretty weird. But I guess I'd first look at the rawlog output of what the clients send to Dovecot and what Dovecot sends back. Also are there any error messages in Dovecot's logs? It kind of sounds like the mailbox could be read-only, but there should be no difference between expunging a message that has tags (IMAP keywords) and that doesn't have. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Marking as Read causes Body.Peek on ALL messages - Reasonable?
On Aug 13, 2008, at 12:12 PM, Daniel Watts wrote:
I've noticed when I select a folder of messages (Thunderbird) and
mark them all as read (or unread) it produces the following IMAP
transcript:
Source Destination Info
c.c.c.c s.s.s.s Request: DONE
s.s.s.s c.c.c.c Response: 20 OK Idle completed.
c.c.c.c s.s.s.s Request: 21 uid store 1:20 -Flags (\Seen)
s.s.s.s c.c.c.c Response: * 1 FETCH (UID 1 FLAGS (\Recent NonJunk))
c.c.c.c s.s.s.s Request: 22 UID fetch 2 (UID RFC822.SIZE BODY.PEEK[])
s.s.s.s c.c.c.c Response: * 2 FETCH (UID 2 RFC822.SIZE 1339 BODY[]
{1339}
..
Looks a bit stupid. But I'd guess it's related to Thunderbird being
configured to download mail when reading it and it treats setting
\Seen the same as reading it. Just a guess though.
Is this reasonable? I thought it would be possible to just send a
list of message UIDs and say mark these as /seen /unseen etc.
Sure, it would be possible. Also if Thunderbird really wants to fetch
these messages' bodies, it could have done it using a single command
instead of 20 separate ones..
PGP.sig
Description: This is a digitally signed message part
Re: [Dovecot] sieve - Sendmail process terminated abnormally, exit status 70
Christian Schmidt wrote: Steffen Kaiser, 13.08.2008 (d.m.y): On Tue, 12 Aug 2008, Thomas Harold wrote: Check out /usr/include/sysexits.h what exit code 70 means on your system - 70 is internal software error in Linux. Then check when /usr/lib/sendmail will exit with this code. Deliver will run /usr/lib/sendmail with the uid of the target mailbox, you said virtual user - so you've configured the id in dovecot.conf, I guess. I just had a similar problem caused by the fact that /usr/lib/sendmail was missing. As I'm using exim as MTA, I created /usr/lib/sendmail as a symlink pointing to the exim binary. That was an excellent tip. I started looking closely at /usr/lib/sendmail and following the link chain. Which led me back to /usr/sbin/sendmail.sendmail. Which is probably not the correct sendmail binary to be using when we're running postfix. Apparently, back when I setup this server many months ago, I never installed or ran: # yum install system-switch-mail # system-switch-mail Which switches the links around to point at sendmail.postfix. Once I fixed that, I had to adjust SELinux properties to create a custom profile to allow the sendmail binary to do its work. Thank you both for the pointers, everything is now working properly for vacation auto-responses. (Oddly enough, the broken setup worked with Dovecot 1.0 - and only reared its head after we upgraded to Dovecot 1.1.)
Re: [Dovecot] syslog with PID
On Aug 13, 2008, at 4:59 AM, Heiko Schlichting wrote: I've thought about this before too though. Maybe some kind of a unique cookie could be useful. Sounds good but complicated. The only reason why I try to match login and disconnect is that imap and pop3 does not log a starting message (without enabling debug) and login was the only entry during connect. The following minimal patch solves this in a sufficient way and I suggest to adopt it into the official release: Having these extra Connected log lines are pretty pointless for most people. But this should solve your problem: http://hg.dovecot.org/dovecot-1.2/rev/29b623366e1e With that patch you can do for example: login_log_format_elements = user=%u method=%m rip=%r lip=%l %c mailpid=%e The patch probably doesn't apply to v1.1 code tree though. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] RfE: use HOSTNAME environment variable in hostpid_init()
On Aug 13, 2008, at 4:42 AM, Heiko Schlichting wrote: All parts of dovecot except deliver uses the result of hostpid_init() in src/lib/hostpid.c as a hostname which only asks gethostname(). deliver honours environment variable HOSTNAME in src/deliver/ deliver.c: getenv(HOSTNAME); and uses the hostname of hostpid_init() as a fallback. Well, this is actually trying to use the hostname setting from dovecot.conf. The settings just happen to get passed in environment. Wouldn't it be consequent to evaluate the environment variable HOSTNAME in hostpid_init() with fallback to gethostname() and remove the special behavior from deliver? The dovecot master process should export $HOSTNAME to all childs in this case. Another alternative would be to make the hostname setting global in dovecot.conf and have imap processes use that. Dovecot uses the hostname as part of Maildir file names and gethostname() is not fully qualified on all operating systems (it is on IRIX but not on Linux). But unqualified hostnames are not unique and it is frequent that dovecot hosts are named mail oder imap as hostname. To distinguish between mail.domain1.com and mail.domain2.net would be easier if dovecot evaluate $HOSTNAME in all case not just deliver as program imap can create Maildir files as well. I always thought that hostnames were unique within the installation everywhere even without the domain part, and the domain would just waste disk space in the filenames (and dovecot-uidlist).. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Marking as Read causes Body.Peek on ALL messages - Reasonable?
I've noticed when I select a folder of messages (Thunderbird) and
mark them all as read (or unread) it produces the following IMAP
transcript:
SourceDestinationInfo
c.c.c.cs.s.s.sRequest: DONE
s.s.s.sc.c.c.cResponse: 20 OK Idle completed.
c.c.c.cs.s.s.sRequest: 21 uid store 1:20 -Flags (\Seen)
s.s.s.sc.c.c.cResponse: * 1 FETCH (UID 1 FLAGS (\Recent
NonJunk))
c.c.c.cs.s.s.sRequest: 22 UID fetch 2 (UID RFC822.SIZE
BODY.PEEK[])
s.s.s.sc.c.c.cResponse: * 2 FETCH (UID 2 RFC822.SIZE 1339
BODY[] {1339}
..
Looks a bit stupid. But I'd guess it's related to Thunderbird being
configured to download mail when reading it and it treats setting
\Seen the same as reading it. Just a guess though.
It becomes a bit of a problem when done in a folder of 40,000 messages -
locks up the program for ages and hammers the server a bit too.
Is this reasonable? I thought it would be possible to just send a
list of message UIDs and say mark these as /seen /unseen etc.
Sure, it would be possible. Also if Thunderbird really wants to fetch
these messages' bodies, it could have done it using a single command
instead of 20 separate ones..
Perhaps it does it one by one incase there are a lot of messages but
it could chunk them at least.
Timo - thanks as always for your help with this, particularly as it is
slightly OT.
Ever thought about writing your own mail client? =) Perhaps
LightningDove! The more I use Thunderbird the more I find it totally
insane. Still better than going back to Outlook (thought not tried 2007
yet).
Dan
Re: [Dovecot] Marking as Read causes Body.Peek on ALL messages - Reasonable?
On 8/13/2008, Daniel Watts ([EMAIL PROTECTED]) wrote: The more I use Thunderbird the more I find it totally insane. Did you file a bug report? I imagine this is something that could use some fixing, and there is a much higher chance that the TBird devs will listen than most other clients... I actually like TBird quite a bit, although some things really bug me (like the lack of a proper Signature Manager, the inability to tell it to use the same Server Settings for sending as receiving, etc)... -- Best regards, Charles
Re: [Dovecot] [PATCH] Allow GSSAPI to work with multihomed hosts
On Aug 12, 2008, at 2:28 AM, Jason Gunthorpe wrote: I choose to just use the magic configurable: auth_gssapi_hostname = $ALL rather than introduce more configurables Yes, the less different settings there are the better. :) Committed to v1.2 tree: http://hg.dovecot.org/dovecot-1.2/rev/9ca5e8f66d10 + auth_request_log_info(request, gssapi, +Using all keytab entires); I'm beginning to wonder about the logging in the code though. To me it looks like all of these should rather be log_debug instead of log_info. And I don't see any log_infos for logging why the user login actually failed (does gssapi even tell anything about it?). Or debug logging about what the usernames are when trying to log in. And the GSSAPI errors probably should be logged with log_info instead of log_error, because they probably aren't errors that the sysadmin can do anything about, but rather some client misconfiguration or a client bug (at least after the initial configuration is done and working). Any thoughts on those issues? PGP.sig Description: This is a digitally signed message part
[Dovecot] 1.1.x problems
Hi, since upgrading to 1.1.x i still have this bugs Next message unexpectedly lost I read this Maillist some months ago and i believe that im not the only one... The Panics and crashes in 1.1.1 are gone by upgrading to 1.1.2 but these Next message unexpectedly lost bugs are really annoying if users can't download complete mails. is there something new to this issue. all recommend changes did not help. The only thing i could do is downgrade to 1.0.15 and all works fine. i can not keep the 1.1.x versions in my repo untill this case is solved. :( Thanks Andre
Re: [Dovecot] 1.1.x problems
On 8/13/2008 3:22 PM, Andre Hübner wrote: since upgrading to 1.1.x i still have this bugs Next message unexpectedly lost I read this Maillist some months ago and i believe that im not the only one... The Panics and crashes in 1.1.1 are gone by upgrading to 1.1.2 but these Next message unexpectedly lost bugs are really annoying if users can't download complete mails. is there something new to this issue. all recommend changes did not help. The only thing i could do is downgrade to 1.0.15 and all works fine. i can not keep the 1.1.x versions in my repo untill this case is solved. :( Output of dovecot -n might help... -- Best regards, Charles
Re: [Dovecot] dovecot sieve sends vacation messages with null envelope sender
on 8-12-2008 7:26 AM CJ Keist spake the following: Josef, This is exactly same situation in our environment as well. I had to make the same change as you stated. Our central virus/spam email gateway also cans any message with out a valid from address in the mail headers. I would think most virus/spam systems will do the same. I understand the concept of having the from empty, namely to keep another automated system from replying back to your vacation reply, but what do we do to keep our vacation replies being canned by anti-spam systems?? Maybe this should be a option in the configuration file to let the Admin decide whether to fill in the from address field, or a [EMAIL PROTECTED] default, or just empty? IMHO if someone's e-mail is so important that you would have to notify people that they are on vacation, then a human should be checking their account everyday. A vacation message just tells me to go somewhere else, which would probably end up being a competitor. If I know the person, I probably already know they are on vacation. If I don't know them, I probably don't care. I will let people have a vacation message, but I will tell them that they might not get delivered. If someone leaves, I alias their mail to their successor, or just delete their account. -- MailScanner is like deodorant... You hope everybody uses it, and you notice quickly if they don't signature.asc Description: OpenPGP digital signature
Re: [Dovecot] 1.1.x problems
Hi, On 8/13/2008 3:22 PM, Andre Hübner wrote: since upgrading to 1.1.x i still have this bugs Next message unexpectedly lost I read this Maillist some months ago and i believe that im not the only one... The Panics and crashes in 1.1.1 are gone by upgrading to 1.1.2 but these Next message unexpectedly lost bugs are really annoying if users can't download complete mails. is there something new to this issue. all recommend changes did not help. The only thing i could do is downgrade to 1.0.15 and all works fine. i can not keep the 1.1.x versions in my repo untill this case is solved. :( Output of dovecot -n might help... ok, i did this some times already and was in contact with timo. may be there are some news in this case. i looked in the archives for similiar problems and saw that some users have this problem with 1.1.x and did not get answers f.i. http://dovecot.org/list/dovecot/2008-August/thread.html 2 guys with this problem, but no solution this is dovecot -n # 1.1.2: /etc/dovecot.conf protocols: imap imaps pop3 pop3s ssl_ca_file: /path/path/*.domainname.com.bundle.crt ssl_cert_file: /path/path/*.domainname.com.crt ssl_key_file: /path/path/*.domainname.com.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login login_greeting: domainname.com mailserver ready. login_process_per_connection: no login_processes_count: 1 max_mail_processes: 100 verbose_proctitle: yes mail_location: mbox:~/mail:INBOX=/var/spool/mail/%u mail_cache_min_mail_count: 30 lock_method: flock mbox_read_locks: dotlock mbox_very_dirty_syncs: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): mail_log mail_plugins(imap): mail_log mail_plugins(pop3): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(pop3): /usr/lib/dovecot/pop3 imap_client_workarounds(default): outlook-idle netscape-eoh tb-extra-mailbox-sep delay-newmail imap_client_workarounds(imap): outlook-idle netscape-eoh tb-extra-mailbox-sep delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): oe-ns-eoh outlook-no-nuls auth default: verbose: yes passdb: driver: shadow userdb: driver: passwd lock_method was already changed to fcntl but no change :( this is my configure-line, i do the packaging on my own: ./configure \ --prefix=/usr \ --sysconfdir=/etc \ --localstatedir=/var \ --mandir=/usr/share/man \ --with-ssl=openssl \ --with-storages=mbox,maildir,raw \ --with-pam \ --with-passwd compiled with no errors/no packagingerrors with gcc 4.1.2 version 1.0.x works without any problems, conf is almost the same, just changes required by 1.1.x Thanks you Andre
[Dovecot] Webmail app ... again.
There was a thread several weeks (or months) ago on webmail applications. I didn't like any of the options and only found few that I did but they were either expensive or not maintained. So ... I raised the question on the Catalyst Framework list about Perl based webmail apps and got nothing usable. It came down to what would it take to build a Perl (Cat) webmail app. Several others on the list got interested so we decided to build it. The main issue I run into is my lack of understanding of the ins-and-outs of the IMAP specification and the features that come with using Dovecot. (Oh yeah. Dovecot was the preferred IMAP server!) Another project I'm working on uses IMAP and what I found was there is a great quantity of Perl modules for dealing with IMAP. A great quantity! Unfortunately many of them assume a deeper knowledge of IMAP than I have so are very confusing. What I'm looking for is a good reference, besides the RFC, on IMAP. Anything out there? Electronic or dead-tree is fine. (A book of Dovecot would be neat too.) TIA, Rod --
Re: [Dovecot] 1.1.x problems
On 8/13/2008 3:59 PM, Andre Hübner wrote: ok, i did this some times already and was in contact with timo. may be there are some news in this case. i looked in the archives for similiar problems and saw that some users have this problem with 1.1.x and did not get answers f.i. http://dovecot.org/list/dovecot/2008-August/thread.html 2 guys with this problem, but no solution Ok, sorry, I just wanted to make sure that the info was there for Timo. I do see that there were no responses - maybe he missed them in the deluge... I'm sorry that I cannot help, so hopefully he will chime in soon... -- Best regards, Charles
Re: [Dovecot] [PATCH] Support GSS-SPNEGO natively
Committed the patch to v1.2 tree with some changes: http://hg.dovecot.org/dovecot-1.2/rev/641d761219a6 PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] [PATCH] Support GSS-SPNEGO natively
On Wed, Aug 13, 2008 at 04:23:46PM -0400, Timo Sirainen wrote: Committed the patch to v1.2 tree with some changes: http://hg.dovecot.org/dovecot-1.2/rev/641d761219a6 What happens when the winbind_spnego and the gssapi_spnego are registered at once? I did not address this because I did not have winbind in my tree at the time.. I imagine that the same 'if' that surrounds the internal ntlm module is needed here.. Ideally though it would be nice if the config file could specify a mapping from SASL name to internal module and NTLM_USE_WINBIND could go away. BTW, I have yet to find anything that uses this SASL mode.. MS did not implement it in even the latest version of outlook, despite authoring the standard. :( Thunderbird has all the machinery to support it through SSPI, but it never parses the SASL name to use the negotiate-sspi module, so it doesn't use it either.. Plus, nobody outside of Windows sspi clients cares about NTLM. Thanks, Jason
Re: [Dovecot] Webmail app ... again.
On Aug 13, 2008, at 4:05 PM, Roderick A. Anderson wrote: What I'm looking for is a good reference, besides the RFC, on IMAP. Anything out there? Electronic or dead-tree is fine. (A book of Dovecot would be neat too.) I wrote this a while ago: http://imapwiki.org/ClientImplementation One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] [PATCH] Allow GSSAPI to work with multihomed hosts
On Wed, Aug 13, 2008 at 03:07:55PM -0400, Timo Sirainen wrote: + auth_request_log_info(request, gssapi, +Using all keytab entires); I'm beginning to wonder about the logging in the code though. To me it looks like all of these should rather be log_debug instead of log_info. And I don't see any log_infos for logging why the user login actually failed (does gssapi even tell anything about it?). Or debug logging about what the usernames are when trying to log in. And the GSSAPI errors probably should be logged with log_info instead of log_error, because they probably aren't errors that the sysadmin can do anything about, but rather some client misconfiguration or a client bug (at least after the initial configuration is done and working). Well, I am not an expert on gssapi, but there are definately failures due to administrator misconfiguration and some are the users fault. For instance any failure from obtain_service_credentials is a configuration error. Failures due to service credential mismatch, encryption type mismatch, etc are also configuration errors, but they occure later in the process.. To be honest nobody seems to do a super job of logging kerberos messages. The erro messages from the library are terse and contain no information from the packet. Debugging a service principle name mismatch is a royal pain. The log in my patch probably should be log debug, I just copied the log level from the existing 'Obtaining credentials' message. They are not important unles someone is debugging. Thanks, Jason
Re: [Dovecot] [PATCH] Support GSS-SPNEGO natively
On Aug 13, 2008, at 4:35 PM, Jason Gunthorpe wrote: Ideally though it would be nice if the config file could specify a mapping from SASL name to internal module and NTLM_USE_WINBIND could go away. Well, I renamed auth_ntlm_use_winbind to just auth_use_winbind: http://hg.dovecot.org/dovecot-1.2/rev/1f948670f274 PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Webmail app ... again.
Timo Sirainen wrote: On Aug 13, 2008, at 4:05 PM, Roderick A. Anderson wrote: What I'm looking for is a good reference, besides the RFC, on IMAP. Anything out there? Electronic or dead-tree is fine. (A book of Dovecot would be neat too.) I wrote this a while ago: http://imapwiki.org/ClientImplementation WOW! Thanks. I will use it. It is interesting to see this from the server view point. One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. I've seen several messages on the list that seem to indicate some clients don't maintain the connection. It will be interesting to see what the implications of this. Not sure if it can be done from a HTTP connection as it is suppose to be stateless. Again thanks. Rod --
Re: [Dovecot] 1.1.x problems
On Aug 13, 2008, at 3:22 PM, Andre Hübner wrote: since upgrading to 1.1.x i still have this bugs Next message unexpectedly lost I read this Maillist some months ago and i believe that im not the only one... The Panics and crashes in 1.1.1 are gone by upgrading to 1.1.2 but these Next message unexpectedly lost bugs are really annoying if users can't download complete mails. What happens when this error comes? The client gets disconnected? Or the command just fails? Enable rawlog to see? http://wiki.dovecot.org/Debugging/Rawlog When this happens, does it keep repeating all the time (until you delete indexes)? If so, could you send me the mbox and the index files so I could try to reproduce it? The mbox could be put through http://dovecot.org/tools/mbox-anonymize.pl to remove all the sensitive data from it (but verify that you can still break it with the anonymized mbox..) PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Webmail app ... again.
On Aug 13 2008, Roderick A. Anderson wrote: I've seen several messages on the list that seem to indicate some clients don't maintain the connection. It will be interesting to see what the implications of this. Not sure if it can be done from a HTTP connection as it is suppose to be stateless. Have you investigated Prayer? That's what we use, in a modded version, because it maintains persistent IMAP connections (among other reasons). You could think of it as an IMAP client that happens to be using a web browser to draw back to your screen, as if it were X Windows. It's a sort of clumsy analogy but in the end that's more or less what it amounts to. -- Brian Hayden UMN OIT Internet Services
Re: [Dovecot] Webmail app ... again.
On Aug 13, 2008, at 4:59 PM, Roderick A. Anderson wrote: One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. I've seen several messages on the list that seem to indicate some clients don't maintain the connection. It will be interesting to see what the implications of this. Not sure if it can be done from a HTTP connection as it is suppose to be stateless. It doesn't have to rely on a stateful HTTP connection. You already most likely use some server-stored state based on cookies/whatever. So among that server state you could also keep the opened IMAP connection. I guess the main issue here is that it doesn't work all that nicely if the HTTP requests get redirected to different servers randomly. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Webmail app ... again.
Brian Hayden wrote: Have you investigated Prayer? That's what we use, in a modded version, because it maintains persistent IMAP connections (among other reasons). You could think of it as an IMAP client that happens to be using a web browser to draw back to your screen, as if it were X Windows. It's a sort of clumsy analogy but in the end that's more or less what it amounts to. Beat me to it, and me having spent most of the day porting my patches to Prayer 1.0.x to the new version 1.2.3 :) When we chose Prayer, the persistent IMAP connections were the main selling point (as they were with IMHO a Roxen-webserver-based client written in Pike, a language nobody else seems to have heard of). Prayer keeps an IMAP connection open for up to 10 minutes, AFAIR, but remembers the users session (session id is stored either in a cookie or the URL) for 30 minutes (or two hours on the compose screen). Prayer's user interface is a bit old-fashioned, but is undergoing some serious rewriting at the moment (hence the need for porting my patches). It's server load is very low both on the webmail server and on the IMAP server. I think Dovecot's cacheing may help the more common Webmail apps out there such as IMP and Squirrelmail and probably Roundcube (which does some cacheing of its own), plus there's imapproxy or Dovecot's auth cache if it's authentication that's expensive. Another persistent IMAP Webmail app may be Web-Alpine from UW, but I haven't tried it out yet. If it's expecting to be talking to UW-IMAP it'll need to use persistent connections! Chris -- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, [EMAIL PROTECTED] IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
Re: [Dovecot] Webmail app ... again.
On Wed, Aug 13, 2008 at 04:37:11PM -0400, Timo Sirainen wrote: One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. AlphaMail has this: http://alphamail.sourceforge.net/ It's a webmail written in Perl that uses a C++ middleware which keeps a persistent connection to the backend IMAP server. But I'm afraid the project is dead... Geert pgpC97Qt1PfMU.pgp Description: PGP signature
[Dovecot] Migrating mbox to maildir
On Wednesday, August 06, 2008 10:18 PM -0400 Timo Sirainen [EMAIL PROTECTED]
wrote:
So I guess you're using mbox? There it's safe to delete everything. If
you're using maildir you should keep dovecot-uidlist and dovecot-keywords.
I'm in the process of getting my head around the least painful path to
convert everything to maildir, given the headaches mbox is giving me here.
It looks like I can leave mail_location unset, and use a basic namespace to
force the separator to be the same as for mbox:
namespace private {
separator = /
inbox = yes
}
procmail is used as the LDA on CentOS so I then need to change procmailrc
to direct incoming mail to Maildir, presumably by adding:
MAILDIR=$HOME/Maildir
DEFAULT=$MAILDIR/
It looks like I can do this on a per-user basis by putting that in a user's
.procmailrc until I migrate everybody.
Re: [Dovecot] Procmail versus Dovecot LDA
On Aug 13, 2008, at 12:02 AM, Gerhard Wiesinger wrote: BTW: Timo please fix the bugs regarding deliver and dovecot index bugs as already discussed. Scanning large mailbox folders takes a lot of time. If you need any help just let me know it. I was never able to reproduce the problem myself, and the last time we found a solution to one problem it was a configuration mistake in your end. So I'm a bit afraid if I again spend a lot of time with it we'll only find out that it's a configuration issue again.. Or possibly an issue in filesystem/something. PGP.sig Description: This is a digitally signed message part
[Dovecot] Yea[h]
Timo Sirainen wrote: Yea, ... I've been meaning to tell you that should be Yeah for an informal version of Yes, otherwise it's a very archaic form of Yes or Indeed as in Yea, though I walk in the valley of the shadow of death! I think that's also true of American/Aussie etc. as well ... Best Wishes, Chris -- --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+- Christopher Wakelin, [EMAIL PROTECTED] IT Services Centre, The University of Reading, Tel: +44 (0)118 378 8439 Whiteknights, Reading, RG6 2AF, UK Fax: +44 (0)118 975 3094
Re: [Dovecot] Cyrus vs Dovecot
Mathieu Kretchner [EMAIL PROTECTED] wrote: kbajwa a écrit : Cyrus = 0 Dovecot= 100 I guess you've right but I can't post this answer at Cyrus mailing list. I'm just trying to have my own opinion of imap server and I already have sarcastic answer on the cyrus mailing list ! Stop. What's this? a) crossposing content to the dovecot mailing list b) talking about sarcastic answers when users try to help you saying that migrating from an old cyrus release to a new one is easier then migrating to a new system? c) many users here have described their running configuration to help you. d) starting an advocacy war? What are you trying to do?
Re: [Dovecot] Cyrus vs Dovecot
On 13 Aug 2008, at 10:31, kbajwa wrote: I think you are missing a point which is most important, i.e., what type of support Cyrus vs Dovecot offers. In my experience: Cyrus = 0 Dovecot= 100 As someone who answers many help requests for cyrus (and I'm very far from the only one), I can honestly say I've never seen a requests from you. Perhaps you've had a lot of occasion to ask for help with Dovecot. I'm happy to hear you've gotten that help. Community is a lot of what open source software is about. As for your experience with the cyrus imapd community, perhaps your sample size is too small. Or perhaps you're thinking of paid support? Because I know very well that you can get that for cyrus imap. :wes
Re: [Dovecot] Yea[h]
On Aug 13, 2008, at 5:24 PM, Chris Wakelin wrote: Timo Sirainen wrote: Yea, ... I've been meaning to tell you that should be Yeah for an informal version of Yes, otherwise it's a very archaic form of Yes or Indeed as in Yea, though I walk in the valley of the shadow of death! Hmm. I've never paid attention to that. Grepping my IRC logs I seem to have used that since the beginning. But I can also see a lot of other people are saying yea (but no idea if they're native english speakers). Wikipedia says it's a common misspelling. Perhaps I should try to change it. :) PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Corrupted index cache files
One or more users? Many different users. Post your dovecot -n output? Here's some of it. Not very enlightening. # 1.1.2: /etc/dovecot/dovecot.conf base_dir: /var/run/dovecot/ verbose_proctitle: yes first_valid_uid: 6 last_valid_uid: 5 first_valid_gid: 6 last_valid_gid: 5 mail_access_groups: mail mail_privileged_group: mail mail_location: maildir:~/mail Can you reproduce these easily with my imaptest tool? http://imapwiki.org/ImapTest Some of them. When running imaptest I see these dovecot errors: Corrupted index cache file %s: record continues outside its allocated size Corrupted index cache file %s: record points outside file Corrupted index cache file %s: used_file_size too large Plus I see LOTS of errors from imaptest itself: Error: UIVALIDITY changed: %d - %d Error: user%d[%d]: ...: Header CC changed Error: user%d[%d]: ...: Header DELIVERED-TO changed Error: user%d[%d]: ...: Header FROM changed Error: user%d[%d]: ...: Header IN-REPLY-TO changed Error: user%d[%d]: ...: Header MESSAGE-ID changed Error: user%d[%d]: ...: Header REFERENCES changed Error: user%d[%d]: ...: Header SUBJECT changed Error: user%d[%d]: ...: Header SUBJECT changed Error: user%d[%d]: ...: Header TO changed Error: user%d[%d]: UID %d changed Message-Id Error: user%d[%d]: UID=%d INTERNALDATE changed Error: user%d[%d]: uid=%d ...: BODY changed Error: user%d[%d]: uid=%d ...: BODYSTRUCTURE changed Error: user%d[%d]: uid=%d ...: BODY[%d] size changed Error: user%d[%d]: uid=%d ...: BODY[HEADER] size changed Error: user%d[%d]: uid=%d ...: BODY[TEXT] size changed Error: user%d[%d]: uid=%d ...: BODY[] size changed Error: user%d[%d]: uid=%d ...: ENVELOPE changed Error: user%d[%d]: uid=%d ...: RFC822.SIZE size changed One problem with HFS+ is that hard links are more or less buggy. But v1.1's default settings should include dotlock_use_excl=yes. You maybe should set maildir_copy_with_hardlinks=no, but that shouldn't cause this bug. Changing maildir_copy_with_hardlinks makes no difference. You say that the dovecot errors are harmless because dovecot fixes them, but still there must be some downside, if only a performance hit. Any other info I can provide to help you figure this out?
Re: [Dovecot] Corrupted index cache files
On Aug 13, 2008, at 6:09 PM, Mike Abbott wrote: mail_access_groups: mail mail_privileged_group: mail You probably won't need either of these. And there's no point in setting them to the same value. Can you reproduce these easily with my imaptest tool? http://imapwiki.org/ImapTest Some of them. When running imaptest I see these dovecot errors: Corrupted index cache file %s: record continues outside its allocated size Corrupted index cache file %s: record points outside file Corrupted index cache file %s: used_file_size too large How soon? With what kind of imaptest parameters? I can't reproduce this on my Macbook (OS X 10.5.4, HFS+). Plus I see LOTS of errors from imaptest itself: Error: UIVALIDITY changed: %d - %d Did you test with a single user or multiple users? imaptest's support for testing multiple users is currently a bit broken, but this shouldn't happen with a single user.. Error: user%d[%d]: ...: Header CC changed Error: user%d[%d]: ...: Header DELIVERED-TO changed Error: user%d[%d]: ...: Header FROM changed These shouldn't be happening as long as you use a test mbox file that has unique Message-ID headers. Were you using the recommended dovecot- crlf as the input? You say that the dovecot errors are harmless because dovecot fixes them, but still there must be some downside, if only a performance hit. Any other info I can provide to help you figure this out? Yes, there's a performance hit since all the cached data is lost. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] mbox empty messages in Sent folder
On Wed, Aug 6, 2008 at 4:26 PM, Timo Sirainen [EMAIL PROTECTED] wrote: On Aug 6, 2008, at 6:11 AM, Diego Liziero wrote: On Mon, Aug 4, 2008 at 4:17 PM, Timo Sirainen [EMAIL PROTECTED] wrote: Maybe this helps? http://hg.dovecot.org/dovecot-1.1/rev/8ab845d3c96d It seems so, thanks Timo. With this patch, by now, all sent mails are correctly written in Sent folder, I'let you know if I've just been lucky :) Definitely solved. I asked the most complainig users to test if it's fixed and they say yes. The most affected client was horde/imp webmail. Thanks again Timo. BTW I didn't succeed in reproducing this issue with imaptest, what was the trick to trigger it? I'm not sure if there's an easy way to reproduce it. You'd have to cause the first read to return EAGAIN but the second read that comes only microseconds later to return the entire message. Perhaps if imaptest sent first the APPEND command, then did a small pause and after that sent the message. Mmm.. I tried to comment out the cork part and added a 10% random sleep after sending the command if (!(rand()%9)) usleep(rand()%500); and I started getting the famous Error: IMAP(testdove): FETCH for mailbox INBOX UID xxx got too little data: yyy vs zzz instead. Regards, Diego.
Re: [Dovecot] mbox empty messages in Sent folder
On Aug 13, 2008, at 6:19 PM, Diego Liziero wrote: Mmm.. I tried to comment out the cork part and added a 10% random sleep after sending the command if (!(rand()%9)) usleep(rand()%500); and I started getting the famous Error: IMAP(testdove): FETCH for mailbox INBOX UID xxx got too little data: yyy vs zzz instead. With the latest code? Sounds interesting. Could you send me all the changes as a patch? I probably won't have time to check it until next week though. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Corrupted index cache files
On Aug 13, 2008, at 6:09 PM, Mike Abbott wrote: Can you reproduce these easily with my imaptest tool? http://imapwiki.org/ImapTest Some of them. When running imaptest I see these dovecot errors: Corrupted index cache file %s: record continues outside its allocated size Corrupted index cache file %s: record points outside file Corrupted index cache file %s: used_file_size too large You could try if it makes a difference to set mmap_disable=yes or lock_method=flock. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Webmail app ... again.
Geert Hendrickx wrote: On Wed, Aug 13, 2008 at 04:37:11PM -0400, Timo Sirainen wrote: One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. I'm sure native support would be better, but how is this different from using Squirrelmail with IMAPProxy? -- Daniel
Re: [Dovecot] Webmail app ... again.
On Aug 13, 2008, at 6:53 PM, Daniel L. Miller wrote: Geert Hendrickx wrote: On Wed, Aug 13, 2008 at 04:37:11PM -0400, Timo Sirainen wrote: One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. I'm sure native support would be better, but how is this different from using Squirrelmail with IMAPProxy? Squirrelmail still keeps asking for new mail etc. The only thing imapproxy helps for is that the IMAP process stays alive longer. Maybe helps a bit since there's less forking, but that's about it. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Corrupted index cache files
How soon? With what kind of imaptest parameters? I can't reproduce this on my Macbook (OS X 10.5.4, HFS+). I ran imaptest for just a few minutes and saw all these errors in that time. Default imaptest parameters except for user/host names etc. Error: UIVALIDITY changed: %d - %d Did you test with a single user or multiple users? imaptest's support for testing multiple users is currently a bit broken, but this shouldn't happen with a single user.. This error happens with clients=1, clients=10 and clients=100. Error: user%d[%d]: ...: Header CC changed Error: user%d[%d]: ...: Header DELIVERED-TO changed Error: user%d[%d]: ...: Header FROM changed These shouldn't be happening as long as you use a test mbox file that has unique Message-ID headers. Were you using the recommended dovecot-crlf as the input? I passed in your dovecot-crlf file to imaptest, but the message store was cloned from elsewhere first. So you're saying that dovecot only works with special hand-crafted input, and not with real-world mail? Hmmm :) :) Actually I do see plenty of duplicate Message-ID headers in the mail store (not from your input file). I'll investigate that further. Thanks for the tip.
[Dovecot] migrating from specialized qpopper
I'm hoping to migrate to dovecot from qpopper, with the eventual goal of enabling IMAP. However, my qpopper installation is fairly specialized, so I thought I'd ask here to see if anyone could get me headed down the best path. Here's the situation. I'm running FreeBSD and have a virtual domains setup with an extremely hacked LDA to do delivery to mailboxes within hashed directories under each virtual domain. Each domain also has a separate password file. This is NOT any virtual domain setup that you'd be familiar with, as it is an in-house creation that is about 9 years old now. I have 8000 users utilizing mailboxes on these domains, so having them reconfigure to use realms/domains in their login isn't really an option. With qpopper, we run it from inetd and use hosts.allow to call each instance with special information about the password file location, etc, like this: popper: ALL : twist (/pathto/bin/popper -d -s -T 300 -h %H -p /pathto/www.%H/passwd -x /pathto/www.%H/spool) This allows us to insert the domain (%H) based on what hostname the user is connecting to and change parameters based on that. Would their be any easy way to do this with dovecot? Thanks in advance for your help. Mark
Re: [Dovecot] Search for (any of) multiple terms slow
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Wed, 13 Aug 2008 17:18:37 -0400, Timo Sirainen [EMAIL PROTECTED] wrote: On Aug 13, 2008, at 3:25 AM, Patrick Nagel wrote: When searching for multiple terms at once (any of) with Thunderbird/Dovecot (using FTS Squat indexes), it takes much longer (not double the time, or a bit more than that, but really much, much longer) than when I search for only one term. Yea, OR isn't supported by FTS plugin yet.. I had hoped no-one would notice it ;) I guess I should do something about it. Heh, I see ;) As it is, the FTS plugin is quite useless to us though :( most of the time people are doing a (real world) search, they don't remember the exact word they're looking for. For example they only remember that the mail they're looking for contained English and automobile - or was it Englisch and Automobil, because the mail was in German? So to get a list of all possible mails, they want to make use of the OR connection of search terms... Requires a lot more complexity to the code though.. Couldn't Dovecot just search the index with each single word (as it does now without OR), and then merge the results, filtering out double UIDs? This is probably not the most efficient way to solve this, but it would be definitely *much* faster than not using the indexes at all - and doesn't sound too complex to me. Are there any other not-yet-implemented SEARCH features that don't work with the FTS plugin? Thanks, Patrick. - -- STAR Software (Shanghai) Co., Ltd.http://www.star-group.net/ Phone:+86 (21) 3462 7688 x 826 Fax: +86 (21) 3462 7779 PGP key: https://stshacom1.star-china.net/keys/patrick_nagel.asc Fingerprint: E09A D65E 855F B334 E5C3 5386 EF23 20FC E883 A005 -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: http://getfiregpg.org iEYEARECAAYFAkijga4ACgkQ7yMg/OiDoAUnFwCfXDUDHR+UdlMKkqwn2/nkB61I X3MAn1nK8KVW8RbtzQ6qQGipGb3uMXRa =I8Eh -END PGP SIGNATURE-
Re: [Dovecot] Corrupted index cache files
On Aug 13, 2008, at 7:12 PM, Mike Abbott wrote: Error: UIVALIDITY changed: %d - %d Did you test with a single user or multiple users? imaptest's support for testing multiple users is currently a bit broken, but this shouldn't happen with a single user.. This error happens with clients=1, clients=10 and clients=100. I mean the user parameter, do you use user=something or user=something %d or similar to specify one/multiple different users where imaptest logs on to? Error: user%d[%d]: ...: Header CC changed Error: user%d[%d]: ...: Header DELIVERED-TO changed Error: user%d[%d]: ...: Header FROM changed These shouldn't be happening as long as you use a test mbox file that has unique Message-ID headers. Were you using the recommended dovecot-crlf as the input? I passed in your dovecot-crlf file to imaptest, but the message store was cloned from elsewhere first. So you're saying that dovecot only works with special hand-crafted input, and not with real-world mail? Hmmm :) :) Actually I do see plenty of duplicate Message-ID headers in the mail store (not from your input file). I'll investigate that further. Thanks for the tip. Dovecot doesn't care about any of that, but imaptest's state tracking requires that Message-ID header uniquely identifies a mail. So the imaptest run was done with a mailbox that already contained other data? Can you break if if you try with an empty mailbox where only imaptest starts appending new mails? PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Can't purge folders in Trash with listescape loaded in 1.1.2
Timo Sirainen wrote: http://dovecot.org/patches/1.1/listescape-plugin.c should help. I also committed a couple of fixes to hg to fix the error message. Thanks, the new listescape seems to work.
Re: [Dovecot] Search for (any of) multiple terms slow
On Aug 13, 2008, at 8:52 PM, Patrick Nagel wrote: Requires a lot more complexity to the code though.. Couldn't Dovecot just search the index with each single word (as it does now without OR), and then merge the results, filtering out double UIDs? This is probably not the most efficient way to solve this, but it would be definitely *much* faster than not using the indexes at all - and doesn't sound too complex to me. It would be nice to allow the FTS backend to handle as much of this as possible. For example with Lucene you could merge several AND/ORs into a single query. But maybe it's not worth the trouble to do all that complexity. But at least the OR-only search could be a special case where all the words are sent to backend. In a mixed AND/OR query then it could do several queries and merge the results. And sure, it's not *that* complex but it'll require a few hours of work. Are there any other not-yet-implemented SEARCH features that don't work with the FTS plugin? Shouldn't be. Or perhaps things related to header searches could be better optimized. PGP.sig Description: This is a digitally signed message part
Re: [Dovecot] Cyrus vs Dovecot
On Wed, Aug 13, 2008 at 01:07:34PM -0400, Wesley Craig wrote: On 13 Aug 2008, at 10:31, kbajwa wrote: I think you are missing a point which is most important, i.e., what type of support Cyrus vs Dovecot offers. In my experience: Cyrus = 0 Dovecot= 100 As someone who answers many help requests for cyrus (and I'm very far from the only one), I can honestly say I've never seen a requests from you. Perhaps you've had a lot of occasion to ask for help with Dovecot. I'm happy to hear you've gotten that help. Community is a lot of what open source software is about. As for your experience with the cyrus imapd community, perhaps your sample size is too small. Yeah, there are a few of us here answering help requests, and even helping debugging in some cases. I'd be interested to see where that '0' comes from too. Still, I think Cyrus and Dovecot are the best two imap servers out there, so it's going to be a question of which integrates best with your usage pattern. For a small server, starting with no experience in either, I would probably choose Dovecot. Now that I know Cyrus inside out, back to front, warts and all - well, I'd choose Cyrus because I know how to make it play nice. It's more of a total system in itself though, that you write support stuff around. Dovecot integrates more with other tools in a unix-daemon'y way. Enjoy, Bron ( now if someone came along with a compelling competitior for SASL... )
Re: [Dovecot] Procmail versus Dovecot LDA
Hello Timo! A configuration issue as we had with the config discussed below can be definitly barred. locate only finds somehting in /home/gerhard/Mail/.imap/ which is correct. Config: ~/.imap 3 ls -l total 0 lrwxrwxrwx 1 gerhard users 30 Jun 13 22:30 INBOX - /home/gerhard/Mail/.imap/INBOX # Users should have access to the whole home directory mail_location = mbox:~:INBOX=/var/mail/%u deliver section: # Mail folders should be in ~/Mail mail_location = mbox:~/Mail:INBOX=/var/mail/%u How can I make a more consistent configuration with namespaces? To avoid the symlink we might include: INBOX_CACHE_DIR=~/Mail Do you have the warnings patch (inconsitent sizes/timesptamps between index and filesystem) running at your servers, too? Do you get warnings? Maybe you can include these warnings in the normal release and some other people have the same problems. I think the used filesystem ext3 is rock solid. Ciao, Gerhard -- http://www.wiesinger.com/ On Wed, 13 Aug 2008, Timo Sirainen wrote: On Aug 13, 2008, at 12:02 AM, Gerhard Wiesinger wrote: BTW: Timo please fix the bugs regarding deliver and dovecot index bugs as already discussed. Scanning large mailbox folders takes a lot of time. If you need any help just let me know it. I was never able to reproduce the problem myself, and the last time we found a solution to one problem it was a configuration mistake in your end. So I'm a bit afraid if I again spend a lot of time with it we'll only find out that it's a configuration issue again.. Or possibly an issue in filesystem/something.
Re: [Dovecot] Webmail app ... again.
On Aug 13, 2008, at 4:03 PM, Roderick A. Anderson wrote: Daniel L. Miller wrote: Geert Hendrickx wrote: On Wed, Aug 13, 2008 at 04:37:11PM -0400, Timo Sirainen wrote: One thing that would be nice, that pretty much no webmail does, is to keep a stateful connection open all the time (or at least some of the time) instead of creating tons of short-lived connections that ask the same stuff over and over again. With a stateful connection you could basically run IDLE and wait for changes there instead of asking all the time is there new mail? is there new mail now? what about now?. I'm sure native support would be better, but how is this different from using Squirrelmail with IMAPProxy? Very interesting. I was thinking of hos to do something similar to IMAPProxy. Now I may not have to reinvent that wheel. Seems to me you could use something like mod_perl to have state- keeping processes running that can keep the connections alive, going IDLE after 10 seconds or so after the last request they got. You can limit the number of open connections from any given process with LRU queuing, but I don't have suggestions on how to tie which process gets which request. Perhaps by writing a middle-layer service that all the processes talk to? But the big killer is scaleability and handling multiple servers, which is why some sort of front end like IMAPProxy are attractive. Good luck Sean
