[Dovecot] case (non-)sensitivity for folder names
Is IMAP supposed to be case sensitive or case in-sensitive? It seemed it would be case sensitive because I've had different cases of folders. But today I found I had two folders Spam and spam, with directories .INBOX.Spam and .INBOX.spam on the server. Messages existed in each directory on the server and they were different. The messages could be read from Evolution. However, the list of messages had info mixed up between the folders. I then selected all messages in one of the folders and trashed them (click on trash icon). Messages disappeared in BOTH folders. On the server, after a couple minutes delay, messages in BOTH directories were flagged T. That doesn't seem right. But I don't know enough about what IMAP is supposed to do, or on which end the names of folders get remapped into directory names, to even guess which is messed up here. I do know Evolution keeps a track of what it thinks the folder has, because in the past I have just rm -fr on a folder directory and Evolution still showed each message in its list, although it could not access them. Anyone know what's the deal with that? Unfortunately, there seems to be no operation in Evolution to tell it to discard its own cache. I'm also wondering if life would be any better if I used Thunderbird. -- sHiFt HaPpEnS!
Re: [Dovecot] case (non-)sensitivity for folder names
On Wed, Jan 5, 2011 at 12:05, Timo Sirainen t...@iki.fi wrote: On Wed, 2011-01-05 at 10:09 -0500, Phil Howard wrote: Is IMAP supposed to be case sensitive or case in-sensitive? Case sensitive, except for INBOX. (Or if the server is using case-insensitive filesystem then they're case-insensitive.) So what will Dovecot do when both directories exist: .INBOX.Spam and .INBOX.spam It seemed it would be case sensitive because I've had different cases of folders. But today I found I had two folders Spam and spam, with directories .INBOX.Spam and .INBOX.spam on the server. Messages existed in each directory on the server and they were different. The messages could be read from Evolution. However, the list of messages had info mixed up between the folders. I then selected all messages in one of the folders and trashed them (click on trash icon). Messages disappeared in BOTH folders. On the server, after a couple minutes delay, messages in BOTH directories were flagged T. That doesn't seem right. My guess is that Evolution messes this up. And it might be confused by odd data it gets due to both above directories existing. Unfortunately, there seems to be no operation in Evolution to tell it to discard its own cache. rm -rf ~/.evolution/mail/imap/ Thanks. I'll give that a try soon. -- sHiFt HaPpEnS!
Re: [Dovecot] case (non-)sensitivity for folder names
On Wed, Jan 5, 2011 at 13:20, Timo Sirainen t...@iki.fi wrote: On 5.1.2011, at 19.33, Phil Howard wrote: On Wed, Jan 5, 2011 at 12:05, Timo Sirainen t...@iki.fi wrote: On Wed, 2011-01-05 at 10:09 -0500, Phil Howard wrote: Is IMAP supposed to be case sensitive or case in-sensitive? Case sensitive, except for INBOX. (Or if the server is using case-insensitive filesystem then they're case-insensitive.) So what will Dovecot do when both directories exist: .INBOX.Spam and .INBOX.spam Same as if they were called .INBOX.Spam and .INBOX.HelloWorld. They are two completely different mailboxes. But if for INBOX names are case IN-sensitive, then there will be confusion because anything obeying that concept would see Inbox spam and Inbox Spam as being the same box. There are different ways to deal with case IN-sensitive name matching. A lame way is to translate all names to a specific case and do a lookup at the OS layer (which on Unix is case sensitive, so which name is found, if any, depends on how the case-ify is done). A somewhat smarter way is to scan the list of names and case-ify each, and compare that the case-ified search name (where they are case-ified the same way, example always upper, or maybe always lower). When a match is found, that leaves two alternatives: either stop there and use that name (which means which is used depends on the order names were obtained), or continue for more. If continuing for more and more are found, are they merged is the next question. The logic can get more convoluted with this. The behavior I have seen, through Evolution (I just installed Thunderbird and will be trying that in a while), is that when I had 3 existing, foo, Foo, and FOO, all under inbox (seen as .INBOX on the server, and as Inbox in the client), there were various situations of getting things mixed up among them as if accessing one would really access another, though it was unclear which. But I definitely ran into the case where when I had ONE message in each of foo, Foo and FOO, and deleted the message in foo only, it deleted one message in all three, but left the new message count on FOO as (1) even though it showed now messages. The subfolders are definitely NOT kept completely distinct. It's as if some software was merging them, and some was not. Since all 3 did show up, clearly Dovecot did not merge them to LIST them to the client. My boss was using Apple Mail, then tried Thunderbird, and similar effects are happening. I'm wanting to believe that IMAP is just so poorly defined that everyone is doing things just a bit differently, and enough to sort of work, but mess things up. -- sHiFt HaPpEnS!
Re: [Dovecot] case (non-)sensitivity for folder names
On Wed, Jan 5, 2011 at 16:10, Timo Sirainen t...@iki.fi wrote: On 5.1.2011, at 23.04, Phil Howard wrote: But if for INBOX names are case IN-sensitive, then there will be confusion because anything obeying that concept would see Inbox spam and Inbox Spam as being the same box. It's case insensitive only for the INBOX itself, not its children. So INBOX.spam and inbox.spam are the same, but INBOX.Spam isn't the same as either of them. Ah, I misinterpreted the original reference to be recursive. The behavior I have seen, through Evolution (I just installed Thunderbird and will be trying that in a while), is that when I had 3 existing, foo, Foo, and FOO, all under inbox (seen as .INBOX on the server, and as Inbox in the client), there were various situations of getting things mixed up among them as if accessing one would really access another, though it was unclear which. But I definitely ran into the case where when I had ONE message in each of foo, Foo and FOO, and deleted the message in foo only, it deleted one message in all three, but left the new message count on FOO as (1) even though it showed now messages. I'm pretty sure this confusion comes from Evolution internally. Given the above clarification, that appears to be the case. My boss was using Apple Mail, then tried Thunderbird, and similar effects are happening. I'm wanting to believe that IMAP is just so poorly defined that everyone is doing things just a bit differently, and enough to sort of work, but mess things up. It's possible that multiple clients do case insensitive comparing. I doubt many people have actually tested what happens then. I only know Dovecot's code, and there's about zero possibility of it mixing up mailboxes with different casing. Some of this might actually be more a case of how clients handle cases where nothing is subscribed. Apparently Thunderbird has no such mode, and always subscribes to what it creates. Evolution lets you run unsubscribed, but behaves like it had some kind of phantom subscription state. Some things do clear up when .evolution state is wiped clean. I'm moving over to Thunderbird in the next day or so. -- sHiFt HaPpEnS!
[Dovecot] re-indexing moved mail
I have a huge block of email that was misdelivered into the wrong folder. Moving it where it should be is not just all from one folder to being all of another folder. Basically, the index files need to be rebuilt. Looking in the wiki I cannot find how to do this. It all seems to be about installation, and no administrative tools. Or would it be just as simple as delete certain index files and they would be automagically rebuilt? -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
On Thu, Dec 23, 2010 at 17:08, Willie Gillespie wgillespie+dove...@es2eng.com wrote: Phil Howard wrote: I think this issue has been entirely misunderstood. Have I explained it wrong? I think there's been a bit of confusion here. Everyone is saying similar things in slightly different ways. The IMAP protocol has no way to push to the MUA that a folder (mailbox) has been freshly created. This information must be pulled by the client, ie: LISTing all folders. Given that it appeared to be a request/response class protocol, I was expecting that it had no such push ability. So, say we have an MUA connected and -something- (whether it be deliver or another MUA connected elsewhere) creates a new folder. With IMAP the original MUA has no way of knowing that this just happened. When it tries to create the same folder, the CREATE fails, because the folder already exists. Right. How the MUA handles this situation is up to the MUA. I see a few possibilities: 1) it could ignore the situation and just show an error message to the user* 2) it could do a LIST and get an updated list of folders** 3) it could add the folder to its display*** * Sounds like what your MUA is doing. Yes. ** This could be fine and dandy, but many MUAs use the subscription list (LSUB) instead of showing all the folders (LIST). So just because the MUA now knows the folder exists doesn't mean it will show it to you unless you SUBSCRIBE to it. However, if I am not doing subscriptions, shouldn't it show me ALL folders (per what Charles Marcus said in his message just before yours)? Why would this folder be handled differently if it is showing me all the other folders? *** Whether this means that the MUA auto-SUBSCRIBEs you to the mailbox or not depends on what mode the MUA is running in. It seems like this is what you want your MUA to do instead of #1. No, that is not what I want it to do. What I want it to do is #2 ... and show me ALL the folders, with the new one included from the most recent LIST. It should do LIST as a result of there being an error from CREATE ... to determine if the error was because the folder had been created by other than the MUA. If I were a programmer, #1 would definitely be the easiest to do. Then I wouldn't have to care WHY the CREATE failed, I just show an error message no matter what. But #2 is not really harder. It's another step. I don't think of such logic has harder. To me, hardness of programming is the difficulty level of figuring out what algorithm to use ... e.g. what works and is expected to work. I would do #2. If as a programmer I was trying to make it easier, I'd just not write any of it at all. When I do programming, though, I consider that the effort to meet reasonable human expectations is part of the job/project. If the developer believes humans expect to not be able to get to a folder because it had previously been created by something else, it should at least be informative ... Sorry, you cannot access folders that were created by other than you, without restarting the client. It's just so much simpler, even with the need to do a whole LIST request, to give the human the realistic expectation of seeing the folder show up after a folder creation dialog, regardless if something else created it first. So in answer to the question in the subject, Is this really a user agent issue? Yes. The server is doing nothing wrong according to protocol. I really didn't think it was. But I was wondering if there was some possibility the IMAP protocol had a limitation that completely prevented this (e.g. it wouldn't be in the LIST response, either, or LIST is only allowed once when connecting, or whatever). It appears IMAP is a minimal but reasonable protocol, and provides sufficient means for MUA logic to be reasonable, and Evolution fell short of that. I wanted to be sure that assumption was correct. -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
I think this issue has been entirely misunderstood. Have I explained it wrong? 2010/12/22 Karsten Bräckelmann guent...@rudersport.de: On Wed, 2010-12-22 at 09:34 -0500, Phil Howard wrote: 2010/12/21 Karsten Bräckelmann guent...@rudersport.de: Creating the new mail folder is entirely on the IMAP server side. The MUA (Evolution in your stated case) is irrelevant. If the creation of the new folder fails, it is a server side problem. However, once a new folder has been created (server side, mind you), Evolution won't know about that folder until it is restarted. Probably the same with other MUAs, too. In the worst case, restarting Evo twice should show the new folder. (That is assuming you are not limiting your MUA to subscribed folders only, or, as IIRC is the default, deliver auto-subscribes the user to the just created folder.) Then I would call this a user agent issue. You snipped the part of your OP where you stated that dovecot deliver fails creating a mail folder. This is NOT an MUA issue. Then whose issue it is? There is clearly an issue because an existing folder cannot be accessed by this MUA. If IMAP provides a way for the user agent to discover the folder already exists, then the user agent should do this, at least if an error is encountered when trying to create it. Evolution clearly does not (or doesn't act on knowing if it does). Other user agents, I don't know. But it is simple logic: There is a way for the MUA to discover folders existing, LIST. Existing on the IMAP server, mind you, again. So this is not an MUA limitation. But it would be an MUA issue if the MUA fails to do LIST after an error from CREATE or whatever command it is. Also, MUAs don't care about the full list of folders, if they are in subscription mode. In that case, they ask for subscribed folders, LSUB, not all folders. It was not in subscription mode. Which method do you use? (That's a client option.) Whatever it starts out as. I don't do subscriptions because so far it hasn't needed to do that. I don't want a subset of folders. I want all folders. How would subscription help there? if (create folder $name succeeds) OR (folder $name can be accessed) then set up local reference to folder $name This is unrelated. *sigh* In your case, the creation is server side. And yes, unless you use subscription only, Evo does exactly that on the next start -- ask for all folders on the server. The point is, if it tries to create a folder, and there is a failure, and because a developer should know and understand that an existing folder is a cause for a failure of a create operation, it should try to distinguish this. A user expects the folder to show up in the list when created when there is no error. If the folder already exists, then there is no valid reason to not put that folder in the list now, just as a successful create would do. Since the protocol apparently provides no means to reflect this situation accurately, the MUA needs to have some means to get access to the existing folder. Restarting the MUA is not an acceptable means. else report error about creating folder $name endif Impossible. The error creating the folder occurred on the server, during delivery, which does not even need to be doveot deliver but any number of other MDAs. No. The delivery successfully created the folder. The error happened later when the MUA tried to create the same folder, due to the MUA not being aware the folder had been created during delivery. But in the case of Evolution, because of that failure in the create operation, it did not allow any access to that folder. The existing folder it tried to create did not get added to the list. It could have been had the logic I described been used. Or it could have been had IMAP provided for a better explain error (more often referred to as an exceptional condition in the classic programming methods I originally learned, rather than a pure unexplained error). Still, the MUA could get around that with the suggested logic. Either way, no client needs to be running while mail is being delivered, and often is not. The server just does not even tell the client about a failure, even *if* it potentially could know about that. This was purely a server-side operation and failure, and the client, any client, is not going to be informed about it. Only a successfully created folder will be known to the MUA during the LIST command. If the folder was created by delivery of mail after the MUA was started, and then how do you think an MUA should become aware of that folder ... other than by restarting the MUA which is not an acceptable requirement. That, of course, depends on IMAP having a means to detect if the folder exists (even though the folder was not given in a previous list of existing folders). Trying to open it should be a way to do that test. Yes, the IMAP server does detect new
Re: [Dovecot] Is this really a user agent issue?
On Thu, Dec 23, 2010 at 10:27, David Warden war...@geneseo.edu wrote: Out of curiosity, are you using the autosubscribe feature of Dovecot LDA to automatically subscribe people who whatever new folder the Dovecot LDA automatically creates? It looks like this is the -s flag to LDA binary in 1.1.3+ or lda_mailbox_autosubscribe in 2.0+ (not sure exactly what point release...) I didn't purposely use it. I see nothing in /etc/dovecot/dovecot.conf that suggests that. I see no -s flag on any running binary. There is no -s flag configured for deliver in Postfix's master.cf. So I conclude that answer to be no. I don't have a lot of experience with Evolution but I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. The best you can do is make sure the client gets automatically subscribed to that new folder and hope that the client checks the list of subscribed folders (the LSUB IMAP command) eventually. I would hope that clicking check/get new mail would be enough to trigger that in most clients. I don't know the semantics of this, and would not want to suddenly change things on users. Maybe they are doing subscribes on their own that might highlight what they subscribe to and ignore all else ... and that doing autosubscribe might interfere with that. Evolution and Thunderbird seem to be fine with no subscriptions going on. On startup they seem to find all the folders. The issue I'm encountering (tried only on Evolution since that is what I use right now) is that when I want to access a folder that was created during delivery, I cannot. There is no folder shown and no option to refresh it (unless doing a subscribe to an as-yet unseen folder would do it, after which I could then unsubscribe) short of quitting Evolution and restarting. Doing Send/Receive doesn't refresh (I didn't really expect it to, but I did try it just in case). The most obvious option is doing a right button click on the parent folder and selecting refresh. That didn't add the folder, either, so I conclude it only refreshes the count of emails. So I tried to create it within Evolution. I figured that would at least set up the folder name in the tree on the left panel. But, since IMAP reported a failure to create (because it existed at that time, due to the earlier create during delivery), Evolution just gave up and did not add the folder. I believe what it should do when creating a folder fails is to check to see if that folder already exists, and if so, either add the folder anyway, or ask the user if the already created folder should be added. When programming in C, when I want to create a file in a directory, and want to create the directory if it does not yet exist, what I code is a mkdir() call with error handling that ignores EEXIST. The intended directory name might be something other than a directory. But I know that open() naming a file in that directory to be created, will fail if it wasn't a directory. So I do not need to explicitly stat() the directory to see if it is a directory or not. If open() fails, then I sort out the error to the level that program calls for. Such a program would create the file in that directory whether the directory exists or not. Extending this concept to a program using a protocol like IMAP ... I don't know if that's beyond the ability of the application developer(s) to grasp, or not. But it is the case that I find very little goofy logic in systems and server daemons, and find a lot of that in GUI applications. I just didn't want to jump to even an obvious conclusion in this case, knowing that I did not know details about IMAP (if I had known IMAP, I probably would have come to some conclusion, quickly). -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
On Thu, Dec 23, 2010 at 12:28, Charles Marcus cmar...@media-brokers.com wrote: On 2010-12-23 10:27 AM, David Warden wrote: I know Thunderbird can be difficult when it comes to new folders showing up on the server that it didn't create. Main reason I always uncheck the 'show only subscribed folders' option in the Advanced settings... If folders were automatically subscribed when created, and while the user agent was running and connected to imapd, would there be a notification sent? Or does the user agent in such a case always ask for all subscribed folders and see the new one at that time. It would not help if the new folder did not show up. -- sHiFt HaPpEnS!
Re: [Dovecot] Is this really a user agent issue?
2010/12/21 Karsten Bräckelmann guent...@rudersport.de: Creating the new mail folder is entirely on the IMAP server side. The MUA (Evolution in your stated case) is irrelevant. If the creation of the new folder fails, it is a server side problem. However, once a new folder has been created (server side, mind you), Evolution won't know about that folder until it is restarted. Probably the same with other MUAs, too. In the worst case, restarting Evo twice should show the new folder. (That is assuming you are not limiting your MUA to subscribed folders only, or, as IIRC is the default, deliver auto-subscribes the user to the just created folder.) Then I would call this a user agent issue. If IMAP provides a way for the user agent to discover the folder already exists, then the user agent should do this, at least if an error is encountered when trying to create it. Evolution clearly does not (or doesn't act on knowing if it does). Other user agents, I don't know. But it is simple logic: if (create folder $name succeeds) OR (folder $name can be accessed) then set up local reference to folder $name else report error about creating folder $name endif That, of course, depends on IMAP having a means to detect if the folder exists (even though the folder was not given in a previous list of existing folders). Trying to open it should be a way to do that test. But if the IMAPD process itself can't or won't even try to access that folder because it wasn't in the list when it started, then I see that as a server side issue, or if the protocol doesn't provide any such means to ask for a folder not previously seen, then I see that as a protocol design issue. -- sHiFt HaPpEnS!
[Dovecot] trying to make cmusieve happen globally
In Dovecot 1.1.11 cmusieve is apparently integrated in the Ubuntu 9.10
package dovecot-common since the files are there. I am wanting to
right now just do a very basic test of the setup to see how it runs
before doing more sophisticated steps that could obscure any errors or
bugs (e.g. unit testing).
So I took the following example sieve script from the wiki1 documentation:
require fileinto;
if header :contains X-Spam-Flag YES {
fileinto spam;
}
and recoded it to skip the test and just always do the fileinto step like so:
require fileinto;
fileinto spamtest;
Then I configured my dovecot.conf file (via dovecot-postfix.conf it
really points to) by setting:
protocol lda {
...
mail_plugins = cmusieve
mail_plugin_dir = /usr/lib/dovecot/modules/lda
...
}
and:
plugin {
sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
}
I manually compiled /etc/dovecot/sieve-global.d/spam.sieve so there
also exists /etc/dovecot/sieve-global.d/spam.sievec in the same
directory. Then I restarted both Dovecot and Postfix.
Mail is being delivered, but still only to the same place it always
has. The filesystem shows these sieve files are not being accessed at
all. I've read documents http://wiki1.dovecot.org/LDA/Sieve and
http://wiki1.dovecot.org/LDA/Sieve/CMU to come up with this. But it
just doesn't seem to be happening. So I guess it is a good thing I've
broken my testing down into smaller units. Anything standout that I
misinterpreted about the documentation? If not, I can post the whole
config files (in case there is a conflict with something else I have
done).
FYI, I'm only trying to do a global sieve, not per-user sieve scripts.
I'll tackle per-user stuff at a later date, after the global is
working.
--
sHiFt HaPpEnS!
Re: [Dovecot] trying to make cmusieve happen globally
On Wed, Dec 22, 2010 at 11:46, Anton Dollmaier
antondollma...@aditsystems.de wrote:
plugin {
sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use sieve_before, which is not documented in the wiki.
Changed.
Only this way the sievescript really gets executed.
The global path requires a sieve script per home directory, including the
global file.
I changed sieve_global_path to sieve_before, restarted both
Dovecot and Postfix (though if deliver reads the config file each
time, I guess I should not need to restart for this stuff), but
there's still no effect. Mail is delivered, but still just to the
regular inbox.
Do I need to have per-user sieves set up, too, just to get it to work?
Are there any log messages that I should expect to see when it is
working?
--
sHiFt HaPpEnS!
Re: [Dovecot] trying to make cmusieve happen globally
On Wed, Dec 22, 2010 at 13:27, Thomas Leuxner t...@leuxner.net wrote:
May be a good idea to post some snippets of the actual Postfix delivery
attempts just to make sure it really uses the external LDA. You can also
crank up 'mail_debug = yes' to log more verbosely. IIRC it should show CMU
Sieve warnings too (it's way old). Going forward using Pigeonhole would make
more sense IMHO as it spits out more useful information. Can you also post
the relevant Postfix 'main.cf' and 'master.cf' sections?
If it is not using the LDA, I don't know what it is doing. There have
been several hundred thousand messages going from Postfix to Dovecot
since I installed this a few months ago, all successfully arriving in
the appropriate user INBOX as far as I can tell. Note that I am using
a gmail account for this mailing list because I set this up way back
when I was installing the mail server, and chose to leave this set up
just in case things break. Right now, the mail server is working
fine except for the sieve part behaving as if there is no sieve
part. I just started trying to run it today.
part of main.cf is:
begin
# Use Dovecot SASL for authentication to submit email.
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/dovecot-auth
smtpd_sasl_auth_enable = yes
# Use Dovecot for transport delivery.
virtual_transport = dovecot
dovecot_destination_recipient_limit = 1
virtual_mailbox_domains = cdb:/var/maildb/domains
virtual_mailbox_maps = cdb:/var/maildb/mailbox
end
part of master.cf is:
begin
#
# This is used by virtual_transport = dovecot in main.cf
#
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -a
${recipient} -d ${us...@${nexthop} -f ${sender}
end
Note that the -a ${recipient} part was added on today, due to this
being shown in the http://wiki1.dovecot.org/LDA/Sieve page. I tested
that it did not break ordinary delivery. But things have been working
(sans sieve) for months without it. I just assumed sieve might need
it, or benefit from it.
The filesystem access timestamps are not showing any accesses
happening on either sieve file except my own accesses from the command
line.
I do see the following as suspicious, that the settings in protocol
lda do not show up in dovecot -n at all. But I don't know if they
should or not. Of particular concern was that main_plugin was
missing. But here (domain name redacted with ) is the
whole output of dovecot -n just in case something is relevant (the
protocol lda part of dovecot-postfix.conf follows it):
begin
# 1.1.11: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.31-22-server x86_64 Ubuntu 9.10 ext3
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot/error.log
info_log_path: /var/log/dovecot/info.log
log_timestamp: %Y-%m-%d %H:%M:%S
protocols: imap pop3 imaps pop3s
listen: 172.30.0.28, [fcca::1c], 127.0.0.1, [::1]
ssl_cert_file: /etc/ssl/certs/imapx..crt
ssl_key_file: /etc/ssl/private/imapx..key
ssl_parameters_regenerate: 24
ssl_cipher_list:
ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM
login_dir: /var/run/dovecot//login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(pop3): no
mail_max_userip_connections(default): 10
mail_max_userip_connections(imap): 10
mail_max_userip_connections(pop3): 3
verbose_proctitle: yes
first_valid_uid: 250
mail_privileged_group: mail
mail_uid: vmail
mail_gid: vmail
mail_location: maildir:/home/mail/%Ld/%Ln/mail
mail_debug: yes
mail_executable(default): /usr/lib/dovecot/imap
mail_executable(imap): /usr/lib/dovecot/imap
mail_executable(pop3): /usr/lib/dovecot/pop3
mail_process_size: 768
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(pop3):
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
auth default:
mechanisms: plain login
username_chars:
abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz01234567890...@+
username_format: %...@%ld
verbose: yes
debug: yes
debug_passwords: yes
passdb:
driver: passwd-file
args: scheme=crypt username_format=%Ln
Re: [Dovecot] trying to make cmusieve happen globally
On Wed, Dec 22, 2010 at 13:28, Stephan Bosch step...@rename-it.nl wrote:
Op 22-12-2010 19:12, Phil Howard schreef:
On Wed, Dec 22, 2010 at 11:46, Anton Dollmaier
antondollma...@aditsystems.de wrote:
plugin {
sieve_global_path = /etc/dovecot/sieve-global.d/spam.sieve
Use sieve_before, which is not documented in the wiki.
Changed.
Don't. Check my other e-mail.
I backed that out. It is now back to sieve_global_path.
Mail is delivered, but still just to the regular inbox.
The main question is: did you read and apply the following:
http://wiki.dovecot.org/LDA/Postfix
You mean did I set up Postfix to run Dovecot deliver as the virtual
transport? Yes. It has been doing its thing for several months and
hundreds of thousands of emails. I did add the -a option as described
in the http://wiki1.dovecot.org/LDA/Sieve document.
Parts of some files, and dovecot -n output, in a previous message.
--
sHiFt HaPpEnS!
Re: [Dovecot] trying to make cmusieve happen globally
On Wed, Dec 22, 2010 at 13:28, Stephan Bosch step...@rename-it.nl wrote:
Op 22-12-2010 19:12, Phil Howard schreef:
Do I need to have per-user sieves set up, too, just to get it to work?
No.
I did some debugging. It looks like I do need to have that set up:
begin
deliver(p...@ham.org): 2010-12-22 15:10:03 Error: Per-user script path
is unknown. See http://wiki.dovecot.org/LDA/Sieve#location
end
However, this message itself is broken, because that page (redirects
to wiki1, now) does not have a label called location. Or is it that
the page is broken by not having a label that works for code already
distributed. Whatever.
So I changed it to this, adding the setup for per-user scripts:
begin
plugin {
sieve_global_path = /home/sieve-scripts/DEFAULT.sieve
sieve = /home/sieve-scripts/%u
}
end
With this /home/sieve-scripts/DEFAULT.sieve compiled:
begin
require fileinto;
fileinto INBOX.spamtest;
end
And now I get this:
begin
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: Loading modules from
directory: /usr/lib/dovecot/modules/lda
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: Module loaded:
/usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: maildir:
data=/home/mail/ham.org/phil/mail
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: maildir++:
root=/home/mail/ham.org/phil/mail, index=, control=,
inbox=/home/mail/ham.org/phil/mail
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: cmusieve:
/home/sieve-scripts/p...@ham.org doesn't exist
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: cmusieve: Using sieve
path: /home/sieve-scripts/DEFAULT.sieve
deliver(p...@ham.org): 2010-12-22 15:32:42 Info: cmusieve: Executing
script /home/sieve-scripts/DEFAULT.sievec
deliver(p...@ham.org): 2010-12-22 15:32:42 Info:
msgid=1293049962.4148.20.ca...@lorentz: saved mail to INBOX.spamtest
end
Success!
--
sHiFt HaPpEnS!
[Dovecot] Is this really a user agent issue?
I suspect this is a user agent issue, given that Evolution is flaky in so many areas. When I deliver mail to a subfolder/subbox (e.g. the -m option in the deliver command), and Evolution doesn't know of it, yet, creating it fails, and Evolution still can't get to it. Doing things the other way around (create it in Evolution first, then deliver to it) works fine. Seems to be silliness to me. Just wondered if any Dovecot aspects might be involved ... such as the fact that the case is different (e.g. INBOX in the filesystem, but Inbox in Evolution). -- sHiFt HaPpEnS!
[Dovecot] Moving email between folders outside of IMAP context
I'd like to run a script from cron that will scan certain mailboxes and move selected mail from one folder (usually INBOX) to another folder. The basic idea is to move previously detected spam based on headers prepended to the mail. Since I am not able to figure out how to get Postfix to invoke deliver with the -m option in these cases, and not in others, this is one idea I have come up with to deal with it. The big (hopefully obvious) complication seems to be getting the indexes updated. Is there a way to move mail this way AND then have the indexes updated to reflect that move (maybe even if regenerating the entire index all over again)? -- sHiFt HaPpEnS!
Re: [Dovecot] Moving email between folders outside of IMAP context
On Tue, Dec 21, 2010 at 16:41, Tom Hendrikx t...@whyscream.net wrote: Do not use an external script, but add a global sieve script that sorts messages into some directory, based on your preferred header. How does that get executed. I take it I can't do it via cron. Does it run at deliver time for each mail message? Is it doable in version 1.1.11 (that's what's in my version of Ubuntu)? Do you know which Ubuntu 9.10 package to install (we're stuck on 9.10 until the new server arrives around February). -- sHiFt HaPpEnS!
Re: [Dovecot] Moving email between folders outside of IMAP context
On Tue, Dec 21, 2010 at 17:09, Tom Hendrikx t...@whyscream.net wrote: I am not familiar with Ubuntu specifics, but iirc Stephan Bosch (creator of sieve plugin) has a repo with debian builds of dovecot+sieve. I would need something that is sieve only, as Dovecot is already installed, configured, and running. I'm also considering writing a program to be run by Postfix in place of deliver, which will read the mail to determine how to run deliver. As long as mail comes in on stdin and is written on stdout, it should be relatively easy (pipe(), fork(), maybe setsid(), execve() in child, read() and write() in parent). But this would take some thorough testing to be sure it is safe.
Re: [Dovecot] Moving email between folders outside of IMAP context
On Tue, Dec 21, 2010 at 17:41, Tom Hendrikx t...@whyscream.net wrote:
That software is already written and tested, it is the sieve plugin. If
you're willing to write, test and run a complete new piece of software,
you might as well replace dovecot with a sieve enabled version of the
same package you're running now, and putting your current configuration
in place.
I would not be willing to take the risk of uninstalling and
re-installing Dovecot or Postfix.
Anyhow, maybe sieve is already available in your current package. I'm
not familiar with (contents of) ubuntu packages, so maybe someone more
knowledgeable in this area can step up?
Ah ... apparently so. These files exist, so I'll go do reading on CMU sieve.
meitner/root/x0 /root 299# ( dpkg -l | fgrep dovecot | awk '{print
$2;}' | while read n ; do dpkg -L ${n} | fgrep -i sieve | prefix
package ${n} has ; done )
package dovecot-common has /usr/share/doc/dovecot-common/wiki/LDA.Sieve.txt.gz
package dovecot-common has /usr/share/doc/dovecot-common/wiki/ManageSieve.txt.gz
package dovecot-common has /usr/share/doc/dovecot-common/managesieve.README.gz
package dovecot-common has /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.so
package dovecot-common has /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.la
package dovecot-common has /usr/lib/dovecot/modules/lda/lib90_cmusieve_plugin.a
package dovecot-common has /usr/lib/dovecot/sievec
package dovecot-common has /usr/lib/dovecot/sieved
package dovecot-common has /usr/lib/dovecot/managesieve
package dovecot-common has /usr/lib/dovecot/managesieve-login
meitner/root/x0 /root 300#
--
sHiFt HaPpEnS!
[Dovecot] restricting access by reverse domain name
Is there a feature or plugin to restrict access to IMAP/POP service by the domain name in reverse lookup? It would be even better if this restriction can exclude certain users (e.g. some users can access IMAP from certain networks but other users cannot). -- sHiFt HaPpEnS!
Re: [Dovecot] restricting access by reverse domain name
On Mon, Aug 9, 2010 at 11:18, Timo Sirainen t...@iki.fi wrote: On Mon, 2010-08-09 at 10:55 -0400, Phil Howard wrote: Is there a feature or plugin to restrict access to IMAP/POP service by the domain name in reverse lookup? With v2.0 you could use tcp-wrappers. It would be even better if this restriction can exclude certain users (e.g. some users can access IMAP from certain networks but other users cannot). Then you need to check this during authentication. What passdb do you use? There is http://wiki.dovecot.org/PasswordDatabase/ExtraFields/AllowNets but it doesn't do reverse DNS lookups. For that you'd need to a) use passdb pam with some PAM plugin, b) use passdb checkpassword with your own script, c) add some new code to Dovecot or create a plugin. Most (maybe all) cases will be allowing from all networks except certain ones. The first interest is to block free web mail services that users can access IMAP servers with. I'm using passwd-file: [...] auth default: mechanisms: plain login username_chars: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz01234567890...@+ username_format: %...@%ld verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: scheme=crypt username_format=%Ln /var/maildb/authdir/%Ld/passwd userdb: driver: passwd-file args: username_format=%Ln /var/maildb/authdir/%Ld/passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail -- sHiFt HaPpEnS!
Re: [Dovecot] listen=*, :: broken?
On Wed, Aug 4, 2010 at 10:47, Peer Heinlein p.heinlein+dove...@heinlein-support.de wrote: the default: listen = *, :: should work and should open port 143 on IPv4 and IPv6. On Linux, listening to :: in IPv6 has the effect of listening also on 0.0.0.0 in IPv4, even though only a single socket is involved. If the process binds a socket first to IPv4 then another socket to IPv6, the IPv6 bind will fail with EADDRINUSE (Address already in use). At this point the process is listening to IPv4 but not IPv6. If the process binds a socket first to IPv6 then another socket to IPv4, the IPv4 bind will fail with the same error. But the socket bound to IPv6 will be listening also to IPv4 via 4-in-6. IPv4 peers can connect and will show a peer address like :::: or :::xx.xx.xx.xx depending on how the address gets formatted. On OpenBSD, this does not happen, and a process must bind a socket separately for IPv6 and IPv4 to be able to listen in both address spaces. I don't know how other operating systems do it. But it should be one way or the other. What a program doing such socket binding could (maybe should) do is either understand what OS it is running on and act accordingly, or just bind the IPv6 socket first and treat a failure to bind on IPv4 as a soft error if the IPv6 succeeded. So, if you are on Linux, just use listen = :: and test if you can connect via the IPv4 address. If so, you are good to go. But, if you are on OpenBSD, just use listen = * and test if you can connect via the IPv6 interface address (you can use the IPv6 link-local address if you connect from an IPv6 peer on the same LAN segment and specify the outgoing interface index as part of the address). On another OS, try each way and use what works. Or you can listen on your explicit interface addresses (not :::: or ::.xx.xx.xx.xx) in each address space. -- sHiFt HaPpEnS!
Re: [Dovecot] listen=*, :: broken?
On Thu, Aug 5, 2010 at 09:56, Tom Hendrikx t...@whyscream.net wrote: This is not true for my linux distro at least, which does not listen on ipv4 interface when setting listen = [::] (in dovecot 1.2.13). I need listen = *, [::] to enable support for both ipv4 and ipv6. This sounds like some distro specific implementation/configuration, so saying 'On Linux' is not true. FWIW: I run gentoo linux (in an openvz container). Then I guess this is distribution specific. Or maybe kernel version specific. The C library may also be influencing this. So try all combinations and use what works. -- sHiFt HaPpEnS!
Re: [Dovecot] Ok, I've given up
On Wed, Jul 14, 2010 at 17:49, Steinar Bang s...@dod.no wrote: Phil Howard ttip...@gmail.com: I abandoned sendmail many years ago and haven't looked back. I tried qmail and postfix, and was a lot happier with postfix. I overlooked exim at the time, but from what little I've seen and heard, it should be up there with postfix, making for a tough choice if you didn't have anything to bias your choice (like having used one of them already for a few years). FWIW I switched from sendmail to exim when switching my home server from SuSE to debian back in 2001. And I've really never wanted to switch, neither back to sendmail, nor to anything else. Main reason: I understand the configuration files' logic. I have no experience of MTAs other than sendmail or exim, though... And I fully accept that as a sufficient reason to make a choice. I made the choice of Dovecot, having zero experience with it, because of my experience with Courier. Sadly, making such a choice with zero experience and zero knowledge of either options is the really hard one to make. -- sHiFt HaPpEnS!
Re: [Dovecot] expire
On Tue, Jul 6, 2010 at 11:35, Papp Tamás tom...@martos.bme.hu wrote: /usr/lib/dovecot/expire-tool.sh Is it a script? I'm on Ubuntu 9.10 with Dovecot 1.1.11 and there is no /usr/lib/dovecot/expire-tool.sh though there is a /usr/lib/dovecot/expire-tool file which is an executable binary. marconi/root/x0 /root 381# ls -dl /usr/lib/dovecot/expire-tool -rwxr-xr-x 1 root root 733856 2009-10-15 00:38 /usr/lib/dovecot/expire-tool marconi/root/x0 /root 382# od -Ax -tx4 -w16 /usr/lib/dovecot/expire-tool | head 00 464c457f 00010102 10 003e0003 0001 7fa0 20 0040 000b2b60 30 00380040 0049 001c001d 40 0006 0005 0040 50 0040 0040 60 01f8 01f8 70 0008 0003 0004 80 0238 0238 90 0238 001c marconi/root/x0 /root 383#
Re: [Dovecot] OT: Sieve + Forward as Attachment
On Tue, Jul 6, 2010 at 13:11, Jerry dovecot.u...@seibercom.net wrote: On Tue, 06 Jul 2010 12:55:04 -0400 Charles Marcus cmar...@media-brokers.com articulated: Maybe, but that's not how I read it... only the OP can clarify... I thought it was clear; however, I guess not. Now, I all ready know in advance that certain messages that I receive will need to be forwarded to other associates. The forward as attachment procedure is what I desire to accomplish. I believe this is covered in RFC-822 (RFC-822 MIME encoded attachment). Virtually every MUA that I have used has that as an option as well as normal 'forward' or 'redirect'. I hope that clarifies mu question. I have seen mail clients migrate from true forwarding (but even so, it is a resubmitted message, so technically a new message), to creating a completely new message with the previous message as indented text, and later as an attached document. They may or may not carry over headers. What you want to do is not a form of forwarding and I doubt Sieve would even have the permissions to do it (well, if it can do forwarding then maybe it can inject new mail, which would make me worry). The best I can suggest based on what I understand of this, is to use Sieve to isolate the messages to be acted on, place them in a special folder for that purpose, and have another process come along to pick them up and do the foward with original as attachment thing (many ways to do that, from a cron script on the server to a fetchmail script anywhere).
Re: [Dovecot] expire
Looks like a newer version that I can't help on. Should be plenty others around at some point. -- sHiFt HaPpEnS!
Re: [Dovecot] system v. virtual mailboxes, was Re: Thunderbird problem
On Thu, Jul 1, 2010 at 02:28, Frank Cusack fr...@cusack.net wrote: On 6/30/10 6:11 PM -0400 Charles Marcus wrote: That's just plain silly. Virtual users are extremely simple to setup, no need for MySQL unless you have a bunch. I agree. I am always in favor of virtual users, it just gives you a lot more flexibility. I find system users MORE complicated to setup, actually. You have to worry about system security in addition to IMAP stuff. You always have to refactor things down the road and starting off with system users just makes it more unpleasant. I find a system-user scheme more complicated only when there is not a one-to-one relationship between the system user base and the usernames in one domain. I tend to use a non-system-user scheme more, now, because of things like having different sets of users in different domains, where, if not now, possibly in the future, a LHS will conflict with a system user, meaning I have to map the relationships. In cases where there is one domain and LHS will be the same as the system user forever (about 3 to 5 years in internet time), I'll use system users (with role accounts either forwarded or as real system users, depending on need). Otherwise, the multi-domain, multi-user-set, all stored under one system user, scheme (that I don't like to call virtual because there is nothing virtual about it once you avoid thinking in terms of system users) works quite well. A hybrid, where one or more domains are designated for system users, could still coexist with the multi-domain scheme.
[Dovecot] address extension going into folder
Is it possible with a simple config change to make address extensions, e.g. alice+sa...@example.com, go into a folder, e.g. .INBOX.sales, and do so ONLY if that folder already exists (and just go into INBOX if it does not exist)?
Re: [Dovecot] address extension going into folder
On Thu, Jul 1, 2010 at 08:47, Edgar Fuß e...@math.uni-bonn.de wrote:
With Postfix, I use
mailbox_command = DOVECOT_LDA -n -e -m ${EXTENSION}
where DOVECOT_LDA is the path do dovecot's deliver. The -n switch prevents
creation of the IMAP folder.
See http://wiki.dovecot.org/LDA
But it (-n) will deliver (rather than reject) into the regular INBOX
if the folder for the extension does not exist?
I'm using Postfix, but with dovecot as a transport.
main.cf has:
virtual_transport = dovecot
master.cf has:
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f
${sender} -d ${us...@${nexthop}
dovecot.conf has:
mail_location = maildir:/home/mail/%Ld/%Ln/mail
I wonder if that ${EXTENSION} works in master.cf. I guess I gotta go
back to Postfix for reading. I assumed this would be a Dovecot
problem.
Re: [Dovecot] address extension going into folder
On Thu, Jul 1, 2010 at 10:56, Edgar Fuß e...@math.uni-bonn.de wrote:
But it (-n) will deliver (rather than reject) into the regular INBOX
if the folder for the extension does not exist?
Yes, but isn't that what you were asking for?
Yes, but your statement was about what you did, and I wasn't sure if
your needs matched my needs. I guess it does.
I wonder if that ${EXTENSION} works in master.cf.
No. In master.cf, its ${extension}.
Time to give that a try.
Re: [Dovecot] system v. virtual mailboxes, was Re: Thunderbird problem
On Tue, Jun 29, 2010 at 16:16, /dev/rob0 r...@gmx.co.uk wrote: On Tue, Jun 29, 2010 at 07:28:52AM -0400, Charles Marcus wrote: On 2010-06-28 9:05 PM, Stan Hoeppner wrote: I guess this is different with virtual users than with system users? Are you using virtual or system users Charles? Virtual of course... doesn't everyone? ;) Virtual mailboxes have their place, of course, but they're overused, especially at small sites. I suppose this might be in part because most HOWTOs are for virtual. I recently saw someone asking for help, having set up a simple server with virtual mailbox (yes, singular) and mysql! The querent was trying to add a SECOND account and did not know how! And what do the MySQL proponents say about that? I started into mail on a very small scale, and that approach served me well. I set up Postfix by reading the comments in main.cf; later when I got the idea that I might want POP3 or IMAP, I uncommented lines in inetd.conf (popa3d I think, and uw-imap), and they worked. When kids got old enough to use email, adduser[1] and there they go. It's nice you had that. Most of the mail servers I did in the past didn't even have POP (users logged into a shell account to read mail). Only recently did I even get into IMAP. IMAP was new to me, as was Dovecot (obviously). Not so with Postfix (or Sendmail, for that matter ... but I won't go back there). Oh, and I tried Qmail for a short stint. I didn't get into virtual mailboxes until later, on a job, and when I did, I knew enough to question the wisdom of it. Why did we need this additional authentication database? All our users were using Samba via system accounts too. It could have been all integrated! The advantages I was told of doing it the virtual way were all based on misunderstandings. (One common one: I don't want mail users to have shell access. Giving them a shell of /bin/false and/or setting sshd_config(5) access controls does the job.) If there is one domain, and each user has an email name matching shell names, that's fine. Use system accounts and shells of /bin/false or whatever. But once you have more than one domain, it is possible to have collisions. This can happen with company mergers. User jsm...@companya.com and jsm...@companyb.com could be two different people who need to continue working with their original email addresses, while the former companies operate as business units under a single merged mail server. There are two (or more) different kinds of virtual, too. One involves mapping multiple users of different domains into distinct system usernames which are not necessarily the same as the LHS of their email address. Now a mapping has to be made, and IMAP logins aren't as straight forward for users (one user logs in as jsmith and the other logs in as jsmith2 ... and what if the 2nd J. Smith is the one that takes the reins as CEO). The other is usually called virtual, but I personally don't, since I consider it to be real. I have: mail_location = maildir:/home/mail/%Ld/%Ln/mail I don't see that as any more or less virtual than where every user has a shell account and the config reads: mail_location = maildir:/var/spool/maildir/%Ln I don't think of that as virtual because the user names and domains are unchanged (I'm now counting lower casing the names). I think many if not most of the questions we see on these lists are from people who have made a bad choice of using virtual mailboxes, often as a direct consequence of that choice. Are you referring to all kinds of virtual? Or just some? Which sets of terminology are you using? Personally, I consider it a bad choice when email addresses are mapped to system users, where LHS doesn't always match the shell user name. I consider it bad because of the confusing maintenance involved. The other two methods (usern...@justonedomain with mailboxes literally owned in the filesystem by the user ... or the way I do it now with multiple domains and the mailboxes literally owned in the filesystem by a designated role system user) I consider to be OK. Email grew up with Unix, so it's no accident that Unix shell usage has very nice integration with email. Probably a lot of the folks reading this list would not even need an IMAPd if they knew more about these things. And it also grew up working with either one domain, or multiple domains having a completely joint user set. But mail can also function just fine when the MAIL USERS are completely isolated from the SYSTEM USERS. That doesn't mean doing this makes sense for everyone. But it can make sense for many (multiple domains and disjoint username sets). I often encounter frustrated newbies who tried to do the whole thing all at once. It makes much more sense to start off small, throw in the relational databases later, learning the finer points of how to manage your OS along the way. The secret is that you can have a fully-functional mail server with very little
Re: [Dovecot] %d/%n doesn't work... why?
On Mon, Jun 21, 2010 at 10:32, William Ottley williamott...@gmail.com wrote: I have attempted to force dovecot to put into a sub directly, without using the %d, but that doesn't work either: #mail_location = mbox:/var/vmail/%d/%n mail_location = mbox:/var/vmail/pplsnet.com/%n I wonder if you are trying to do something like I'm doing now on version 1.1.11 (newer than yours but still an old version): # 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31-19-server x86_64 Ubuntu 9.10 ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 imaps pop3s listen: 172.30.0.24, [fcca::18], 127.0.0.1, [::1] ssl_cert_file: /etc/ssl/certs/imap.REDACTED.DOMAIN.crt ssl_key_file: /etc/ssl/private/imap.REDACTED.DOMAIN.key ssl_parameters_regenerate: 24 ssl_cipher_list: ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM login_dir: /var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 verbose_proctitle: yes first_valid_uid: 250 mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:/home/mail/%Ld/%Ln/mail mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 768 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login username_chars: abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz01234567890...@+ username_format: %...@%ld verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: username_format=...@%ld /etc/mailauth/deny deny: yes passdb: driver: passwd-file args: username_format=%Ln /etc/mailauth/%Ld/deny deny: yes passdb: driver: passwd-file args: scheme=crypt username_format=...@%ld /etc/mailauth/passwd passdb: driver: passwd-file args: scheme=crypt username_format=%Ln /etc/mailauth/%Ld/passwd userdb: driver: passwd-file args: username_format=...@%ld /etc/mailauth/passwd userdb: driver: passwd-file args: username_format=%Ln /etc/mailauth/%Ld/passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail
Re: [Dovecot] Ok, I've given up
On Thu, Jun 17, 2010 at 04:46, Chuck McManis chuck.mcma...@gmail.com wrote: So SMTP hasn't changed much in 30 years ;-) I'd be interested in what you consider a 'modern' MTA. I've looked pretty thoroughly at sendmail, postfix, and qmail and of the three qmail is fairly reliable. Not sure what makes a particular MTA more 'vulnerable' to spam. I don't run an open relay and I generally find barracuda central a decent rbl source. Between that and using tcpserver to simply not accept connections from zombies spam hasn't really been an issue. I abandoned sendmail many years ago and haven't looked back. I tried qmail and postfix, and was a lot happier with postfix. I overlooked exim at the time, but from what little I've seen and heard, it should be up there with postfix, making for a tough choice if you didn't have anything to bias your choice (like having used one of them already for a few years). I prefer to avoid DJB's the code is the comment code because it's too hard to maintain. Hard to maintain == risk of breaking it, IMHO. But I do like DJB's CDB concept. Provide a system that gives shell and email service to a dozen users, hosts perhaps 15 or so mailing lists, provides DNS for 20 - 30 machines. Preferred OS and what makes it the one you choose? Preferred MTA and what makes it the one you choose? Name service? ssh implementation? Hardware? I'd prefer Slackware or OpenBSD based on the simplicity. But I do/have run some things on CentOS, Debian, Fedora, FreeBSD and Ubuntu, for various reasons where those get me going faster, or have what I need, including my current mail server on Ubuntu (for a faster get a startup started where needs were not well defined), which will be migrated to Slackware, maybe in early '11. Postfix is my 1st choice due to experience, but Exim seems to be a fine next choice. My authoritative DNS runs on NSD3, and my caching DNS runs on BIND9. They are run on different IPs on the same machines (5, later to be 6, instances of each). Local hidden zones are on BIND9, but I don't have to do a split horizon to have it. OpenSSH. x86_64 machines because it's COTS. The mail server is running on a pair of 500GB drives in RAID1. Next machine will probably be more drives and 1TB each. I'm looking for a RAID controller than can do a 3-way or 4-way mirror. I also rsync everything to another box every hour and working on setting up a delta archive from that backup.
Re: [Dovecot] Logout after SSL/TLS negociation
On Wed, Jun 16, 2010 at 05:22, bmfr bastien.murz...@palomanetworks.com wrote: I've set up Dovecot to use my signed SSL certificate and IMAP client is using STARTTLS on port 143 without secure authentication. The TLS negociation process seems to go through ok but once the negociation is finished the client sends an IMAP LOGOUT message; he does not try to authenticate, below is the Dovecot's info log: dovecot: 2010-06-15 15:37:01 Info: auth(default): new auth connection: pid=3932 dovecot: 2010-06-15 15:37:03 Info: imap-login: Aborted login (0 authentication attempts): rip=130.100.32.54, lip=10.0.3.3, TLS I've decrypted the TLS negociation thanks to Wireshark (in SSL preferences I added the certificate's private key to decode the stream between client and server) but unfortunately it does not say why the client is sending the LOGOUT command. Do you have an idea of what could be wrong in my server set up? What debugging process would you advise me to follow? What client are you using? Have you tried other clients to see if they do the same LOGOUT?
Re: [Dovecot] Logout after SSL/TLS negociation
On Wed, Jun 16, 2010 at 10:42, bmfr bastien.murz...@palomanetworks.com wrote: Hi Phil, Thanks for your reply, I'm using a thrid party IMAP client (can not disclose its name) and testing with other IMAP clients work. I'm wondering if you had an idea of what could trigger this LOGOUT msg from the client and if it could be something I could fix from the server side. I know of no such behaviour with clients I've been using (AppleMail, Evolution, Outlook, Thunderbird). So I have to suspect something in that third party client itself. Maybe the people who support it can tell you how to enabled client debugging modes that give more detail than just the IMAP traffic. Apparently it is making the decision to do a LOGOUT. One speculation is that the server certificate was not authenticated via a known CA certificate. Is the server certificate signed by a well known (to the client) certificate authority? Ordinarily, a client encountering this should notify you about it, somehow (popup in GUI, messages in text, syslog for web based, etc). Check such logs, too.
[Dovecot] passwd-file management utility
I'm now putting together tools/documentation to manage/administer the mail server, including, of course, managing users. Round one will be command line tools. In order to avoid re-invention, I was looking around for existing tools. I really cannot find any. So it seems I will be developing them. The first will be the one to manage users in the passwd-file format. So I'm just asking around if anyone has heard of such tools somewhere Google isn't finding (it does find lots of generic pages with the keywords used in other contexts). If not, I'll be building my own (and this will be for passwd-file format for the userdb/passdb),
Re: [Dovecot] Dovecot 1.1.x and 1.2.x differencies
On Tue, Jun 15, 2010 at 11:30, Eric Rostetter rostet...@mail.utexas.edu wrote: Quoting Veiko Kukk veiko.k...@ekp.ee: I prefer don't fix if it isn't broken philosophy. Reasonable, as long as the version you are running is still supported... And if it isn't, you should upgrade. But when? Immediately? Or later when an issue crops up (since you'd likely have to change something, then, anyway).
Re: [Dovecot] Dovecot 1.1.x and 1.2.x differencies
On Mon, Jun 14, 2010 at 07:43, Veiko Kukk veiko.k...@ekp.ee wrote: Hello, I have been using successfully Dovecot 1.1.x for about a year now. It has been very stable. Now I'm uprading that same system to newer and more powerful hardware and I was wondering whether it is good idea or not to switch to Dovecot 1.2.x series. Could anybody direct me to feature comparision document or explain here main differences betweeen thos two branches? Where the option to upgrade to the latest version (not just a newer version), it is almost always the best choice. Advantages of the latest version include not hearing those words it is already fixed in the latest version. If you need to stay back at an older version, you should already know why you need to do so. In my case, I have done so because the first round of servers were being quickly deployed based on Ubuntu 9.10, which is known to not always be friendly to building things from source for which it also has packages. The next round will be more slowly built, based on Slackware 13.1 (or so), and use the latest versions of all major software in use, built from developer released source.
Re: [Dovecot] failed to store into mailbox 'INBOX': Not enough disk space. with large mail
On Thu, Jun 10, 2010 at 23:06, Stan Hoeppner s...@hardwarefreak.com wrote: Charles Marcus put forth on 6/10/2010 10:19 AM: On 2010-06-10 3:31 AM, Stan Hoeppner wrote: BTW, anyone who has a 44MB INBOX should be slapped repeatedly about the cranium and then educated about POP and IMAP, and how each should be used. So, Stan, how much will you charge to come do the honors for me? It will probably be in the neighborhood of 50 users with Inboxes over 1GB, some over 5GB... O-M-G!? We may as well skip the punishment/re-education of these folks and march them straight to the firing squad. ;) You're talking just _Inbox_ of those sizes right? No sub folders? If so, yes, definitely to the firing squad they go. :) How about just autodeleting INBOX after N days? I think I might just go ahead and automate that on the server ... but just move the deleted mail to a real Trash folder (maybe with a different name). Hmmm ... what to pick for N.
Re: [Dovecot] failed to store into mailbox 'INBOX': Not enough disk space. with large mail
On Thu, Jun 10, 2010 at 11:54, Jakob Curdes j...@info-systems.de wrote: Am 10.06.2010 17:19, schrieb Charles Marcus: On 2010-06-10 3:31 AM, Stan Hoeppner wrote: BTW, anyone who has a 44MB INBOX should be slapped repeatedly about the cranium and then educated about POP and IMAP, and how each should be used. So, Stan, how much will you charge to come do the honors for me? It will probably be in the neighborhood of 50 users with Inboxes over 1GB, some over 5GB... Same here, we run a mailbox_size_limit of 700MB and have several users regularly hitting the limit. Can't really slap them as they would reply on google I have 7 GB mail space. JC Create a folder for them called Google and tell them to move mail from their INBOX over to the Google folder because it has lots more space :-)
[Dovecot] Evolution and Thunderbird do different things?
I'm trying both Evolution and Thunderbird on my IMAP server, and find that there are differences in how some things are done, between clients. Shouldn't there have been a standard way to do these things in the IMAP protocol? The first thing I noticed is that when deleting email from one client, it puts a T on the file name, leaving it where it is, and from the other client, it moves the mail to a .Trash directory on the server (and created a 2nd Trash folder ... so now I have 2 Trash folders, one with some deleted mail in it, and the other with some deleted mail in it). I'm not saying Dovecot has a problem here. But maybe IMAP the protocol does for not having a standard way to do things, and these clients, for not doing it the same way (if there is some standard somewhere). Any IMAP experts here know what the story is with this?
Re: [Dovecot] Evolution and Thunderbird do different things?
On Wed, Jun 9, 2010 at 12:07, Tom Hendrikx t...@whyscream.net wrote: The IMAP protocol does not define folder names and such. Servers and clients only know how to create/remove/rename/relocate folders and files, and some other basics. The names that are used by default, is a choice of the user (mostly the default settings in the users' client). I was afraid of that. A standard for syntax but not as much for semantics at a higher level. 'Trash' is the default trashcan folder in thunderbird, but in MS Outlook it's 'Deleted Items' (not even mentioning differences related to locale settings). When interpreting your experience with Evolution (never used it myself), I guess that it doesn't use the trashcan folder concept at all, but in stead flags a message as 'Trash', in the same way that you would set 'Seen' or 'Important' flags, and treats these message different from a UI perpective. AFAIK only the INBOX is a well-known default (and maybe even part of some RFC). But that would mean there is some mechanism in IMAP for these flags. Dovecot is attaching the flag 'T'. But what does 'T' mean? If IMAP allows setting flags with arbitrary letters, then 'T' could mean Trash for one client and Terrorist for another client, or no meaning at all for yet another. You could try to set up all clients' prefs to use the same naming scheme, and the same way of trash handling, when possible. That'll be the hard part ... that I was was afraid of. It will require getting everyone to use their clients in the same way, disrupting what they already do. I guess it isn't much of a problem for most people because they rarely share a mailbox between different people with different clients.
Re: [Dovecot] Evolution and Thunderbird do different things?
On Wed, Jun 9, 2010 at 14:02, Tom Hendrikx t...@whyscream.net wrote: Flags was actually the wrong phrase, the correct term is IMAP keywords. The way that Dovecot handles this internally, is described in http://wiki.dovecot.org/MailboxFormat/Maildir . But it has no use to investigate the inner workings of dovecot when all clients see the data through the same interface. Dovecot will tell all clients which keywords your message has. Some clients just treat some keywords 'special'. In https://bugs.launchpad.net/ubuntu/+source/evolution/+bug/13983 (google is a bad company but a great tool!) there is a nice thread about differences of trash implementations between Evolution and 'the others'. Thanks for that link. Good reading and perspective. Recalling now, I think this issue triggered me to ditch Evolution after 2 days of testing some years ago, and into using the same client everywhere. But YMMV... I haven't decided, yet. I've used both in different places off and on for the past few years, as well as text based mail agents in command line environments, plus some webmail systems. But at least it helps to understand better what is going on. And I may well move to Thunderbird at some point.
Re: [Dovecot] Dovecot aspects of fighting spam
On Wed, Jun 2, 2010 at 02:39, Rainer Frey rainer.f...@inxmail.de wrote: On Tuesday 01 June 2010 17:01:16 Phil Howard wrote: I cannot determine how deliver with an empty string give to -m would behave. This is just what I mean: the behavior of deliver -m has absolutely nothing to do with fighting spam. It is the same whether you want to use it for spam- tagged mails or any other reason. Therefore you'll not find the description in thedovecot wiki bysearching for fighting spam. You'll find it by searching for deliver (or LDA). I found the man page (like) document for deliver just fine. I did not limit my search to just the spam keyword. I'm guessing that you are assuming I never found this document: http://wiki.dovecot.org/LDA But now that you know I have read it (and read it long before the previous post), re-read what I said. You should conclude that I am unable to determine fully what -m does from this. It does not describe the behavior if the value given is a null string. This is a concern because in master.cf for Postfix, there is no means to dynamically provide or omit the -m option. You can give it a value from a variable. But it may be necessary to give an empty string value in order to not deliver to a specific folder, and I want to know if that will get the mail into the INBOX. Yes, sieve is mostly (with dovecot: only) used at delivery time (Cyrus IMAP might have support for applying sieve scripts with IMAP, but if so, it is an exception). If sieve had been defined in terms of the mail client uploading scripts via IMAP to a designated special folder, it might have been easier to have users supply their own scripts. But at this point I don't want to have users doing things that will send mail back out (e.g. vacation scripts) unless and until I can ensure that such outbound mail only goes to known recipients (e.g. to avoid backscatter for FN spam with forged sender addresses). This is not my experience. Esp. dovecot and postfix are easily managed from source. Installing the package 'build-essential' and then 'aptitude build-dep dovecot' should provide all you need to compile and install dovecot from source (which is documented in the wiki). My experience with Debian and Ubuntu has several cases of packages being degraded or corrupted after building them from source code. Colleagues have reported the same with Fedora. No such issue has ever happened with Slackware, either to me, or that I have heard of. But, for the time being anyway, I am on Ubuntu without any source builds. For various reasons, I do need to eventually move mail services off the current machine and onto a pair of new machines. At that time, Slackware will be considered, as will OpenBSD. This is why I recommended a combination of different measures. Content detection should come in last in that list, but still can be done at SMTP time (esp. with latest postfix and amavisd-new). I'm all for the combination. But I need to do less rejection and more tagging for the before-queue tests (after-queue tests would limited to tagging, only). That's yet to be explored in Postfix.
Re: [Dovecot] Dovecot aspects of fighting spam
On Wed, Jun 2, 2010 at 10:20, Tom Hendrikx t...@whyscream.net wrote: Did you even try to set this up in your test environment (you have one, don't you?) to explore your own capabilities in finding out the behaviour of this edge case? It sounds trivial to do, but you cannot expect other list members to test this for you. I wish. The mail server was the test environment before the domains were moved over to it. The move was originally planned for June 30, but got expedited to May 25 due to issues with the mail service provider. I will be moving this again probably around October. Hopefully that hardware will be in a couple months before. This is a startup environment. It means there isn't a ton of cash up front to have the kind of environment that makes sense. But it is coming in at some pace, now. I may even get a real desktop computer in July. When you change your attitude a bit more towards this direction, you could even add some valuable input to dovecot by suggesting a man page enhancement describing the behaviour of deliver when hitting an empty string as argument to -m (or at least fill the gap when someone else has the same question and tries google for it). Your latest posts to dovecot (and postfix) mailing lists seem to be only highly speculative and have a lot of lengthy arguing, and not much hands-on. The attitude here is more about get as much done as possible. The mail server is a fraction of what is going on. I work on it when I can. Other things did get dropped the week or so leading up to the expedited changeover. Now that mail is off the problem provider, I do have more breathing time and can actually do some reading. But even so, I'd much rather spend less time reading only what I need, than spend more time reading stuff that includes what I don't need. The reason for that is because I have so much other stuff to read on other projects. Today was spent diagnosing why time servers were refusing to start, and why the monitoring programs on the RAID file servers were not able to access the network. If those problems weren't happening, I'd have been working on the lighttpd (new stuff to read) web servers. A small example describing a more pro-active attitude: phil Hi list! I need to put spam in a specific folder and want to use 'deliver -m' for this. However, when I set up deliver like this in postfix config excerpt omitted, I get the following results that I did not expect when the argument for '-m' was a null string: log excerpt omitted. list Hi phil, could you try suggestion?. Also, when you do this config omitted, you'll most likely get what you want. phil That works great, thanks guys. I'll add you suggestion to the dovecot wiki for future reference. But that's not reflective of reality. First, I was not committed to using deliver -m at all. It was ONE option. And there was even more than one way to use it. Now I did misunderstand Sieve at that point, because I was addressing things in a different order. Sieve was on the back burner because of issues related to user Sieve scripts. I knew I needed to read more about Sieve to address it. But because I didn't connect the concept that Sieve would address this problem, I didn't expedite considering it. Remember, the plate is more than just full; it is way overflowing. It is prudent to manage time the best you can. This means avoiding things that don't seem to be urgent. That is not a perfect process, since one might not know about the perfect solution to everything that lurks in the bowels of the documentation. But I do try to phrase my initial questions in terms of what documentation to read, or of basic planning directions. Planning and reading documentation are too often at odds. It's a hard juggle. The best planning is done by knowing everything. But if you spend all the time to know everything, the deadline is long past by the time the decision can be made. So it's something you have to go back and forth with. Read some. Make some decisions based on that. From those decisions now you have narrowed the field of what is to be read. That means you now have the advantage of less to read. And often the details are lacking in the documentation. Ways to fill that in do include testing (now, for example, I'll have to get and set up a whole new testing environment for email ... something I hadn't expected to need to do for a couple more months) ... or, asking. For example, someone could argue that I should have chosen Exim instead of Postfix, and that if all I had done was read everything about Exim, I'd have know it was better. Well, I'm not going to pass judgment on that. Since I have never used Exim and knew nothing about it (besides that it is another MTA), and have used Postfix some and know some about it, I save time by choosing Postfix. It might not have been the best choice. But making the choice is likely the best time saver because it omits all
Re: [Dovecot] recipient_delimiter
On Fri, May 28, 2010 at 17:30, Charles Marcus cmar...@media-brokers.com wrote: On 2010-05-28 5:01 PM, Phil Howard wrote: On Fri, May 28, 2010 at 16:53, Charles Marcus wrote: The whole purpose of the -n output is to provide clean, easy to read *settings* as seen by postfix (as opposed to comments that are meant for people). So you are saying that this is not meant for people? sigh yeah, so I worded it badly... Of course it is meant for people, but it is meant to show only the bare minimum of what postfix sees as the settings. It is left up to you, the sys admin, to be able to interpret the data as presented, bearing in mind all caveats (ie, that last setting wins)... And that caveat was something I was not aware of, as it is an usual method.
Re: [Dovecot] purge maildir folder after sa-learn
On Sat, May 29, 2010 at 12:03, Eric Shubert e...@shubes.net wrote: That's what I do. I move them either to a (shared) ham or spam folder, and run the script posted earlier, which processes and deletes them. Thanks. I just wanted to be sure such a move would not effect (corrupt) what Dovecot was doing (e.g. indexes and such). Glad it won't. That makes things simpler.
[Dovecot] Dovecot aspects of fighting spam
I see no documents in the Dovecot documents wiki on how Dovecot features can be used in the fight against spam. One hit came up in a search, and it was about how to fight spam in the MoinMoin Wiki itself (e.g. TextCHAs and such). I already asked in another thread about removing message files at the search from a designated learn-spam folder. My next question would be how to automatically place detected spam into a specific folder for the user to access or ignore as they deem fit. That would seem to me to be something for the deliver command and its -m option. But this might be more of a Dovecot and MTA coordination issue, too (e.g. how to get the spam detection done in the MTA ... Postfix in my case ... to become a label of some sort that ends deciding the folder. The -m option might be harder to get it dynamically set since it would be hard coded in master.cf for Postfix (so a variable has to be able to direct it to INBOX in this case). But maybe the standard way of doing this is to shim another program between the MTA (Postfix for me) and the LDA (Dovecot deliver) to dynamically add the -m option on the fly? Lacking any other clever solution, this seems to me to be the way I could make it work. Maybe there is a special header deliver could detect, such as X-Spam: yes? A wiki document on how Dovecot helps (or how to help Dovecot) in the fight against spam, I think, would be helpful.
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 09:54, Frank Elsner fr...@moltke28.b.shuttle.de wrote: On Tue, 1 Jun 2010 09:42:18 -0400 Phil Howard wrote: I see no documents in the Dovecot documents wiki on how Dovecot features can be used in the fight against spam. One hit came up in a search, and it was about how to fight spam in the MoinMoin Wiki itself (e.g. TextCHAs and such). Fighting against SPAM is the MTA's job. Dovecot's job is to provide access to the mailboxes for the users. Dovecot would still be involved ... getting detected spam into a folder for users that don't want false positives to be deleted, and a feedback folder for recycling false negatives into detection training.
Re: [Dovecot] recipient_delimiter
On Tue, Jun 1, 2010 at 09:56, Charles Marcus cmar...@media-brokers.com wrote: On 2010-06-01 8:39 AM, Phil Howard wrote: On Fri, May 28, 2010 at 17:30, Charles Marcus wrote: Of course it is meant for people, but it is meant to show only the bare minimum of what postfix sees as the settings. It is left up to you, the sys admin, to be able to interpret the data as presented, bearing in mind all caveats (ie, that last setting wins)... And that caveat was something I was not aware of, as it is an usual method. That's why reading man pages is usually a good idea... ;) Yes. And if Postfix were not such a large and complex MTA (not that others aren't), and if the complexities of getting things working between Postfix and Dovecot were not so hard (if Postfix and Dovecot had supported a common file format for listing users and their crypt passwords, this would have taken a lot less of my time), I might have had time to read it all. Unfortunately, working at a startup does not afford such luxury. Postfix's great advantage is that it supports so many ways to do things. Postfix's great disadvantage is that it supports so many ways to do things. I should have also just blown off all the people trying to get me to use SQL and such, instead of debating them (which was just a waste of time).
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 10:04, Jerry dovecot.u...@seibercom.net wrote: The fight against SPAM is NOT Dovecots responsibility. Whose responsibility is it to get detected spam delivered into the correct folder? Dovecot deliver is typically used to deliver into the Inbox(es). It would be involved. Note that I am not saying responsibility. There are features that can be used (or there aren't, as the case may be). Documenting them would be helpful. Dovecot has documentation for many other things. If it had documentation on ... dealing with (I'll use that term instead of fighting, as maybe that will be clearer to you) ... spam aspects (like folder selection aspects), that would be helpful. The fight against SPAM is NOT Dovecots responsibility. There are numerous applications that work with your MTA for that purpose. I suggest that you investigate this further. You might want to start here: http://www.ijs.si/software/amavisd/ They also have their own mailing list that you can use to answer any questions that you might have. Like How do I get Dovecot deliver to put the spam in the spam folder? ?
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 10:21, Frank Elsner fr...@moltke28.b.shuttle.de wrote: Ok, you obviously have a different environment and requirements. - I do the delivery into special folder by the MTA (exim) and the LDA is not used at all. - detection training is not done. Sorry, my world may be to narrow :-) Each of our worlds is narrow. Some may be wider than others. Programs like Dovecot support a LOT of things, and I can't see how anyone can be using more than a (perhaps significant in some cases) fraction of it. Obviously your choice of Exim for an MTA doesn't relate to my choice of Postfix for MTA. Dovecot supports both and more. So the programs can be a lot wider real world uses. But put all the variety of uses together as a set, and it can be seen that diverse tools are useful. Since you aren't using Dovecot deliver, then you don't need to be knowledgeable about its -m option. I'm curious if that is the only way. I'm still thinking that a shim (program) between Postfix and Dovecot deliver that detects headers, markers, or tags placed on the message by spam detection, and adds a -m Spam option or whatever, might be what I need to do. But I guess you won't need it.
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 10:45, Rainer Frey rainer.f...@inxmail.de wrote: I guess you'll have the mental transfer of fight spam into cause dovecot to perform this and that action yourself. The possible actions are, in general, documented. I cannot determine how deliver with an empty string give to -m would behave. OTOH, I don't know that an empty string would what would even happen, since I have yet to explore how the spam status from one of the spam detections would trickle through to setting up the transport variables in Postfix. I don't even know if -m is considered the appropriate means to deliver to a specific box in the case of junk spam. If it is, obviously someone must be using it. You need to configure the MTA to accept and tag spam messages, e.g. with Amavis D_PASS policy and appropriate tag levels. Then the most flexible way to put the mails into a spam folder is to use the deliver plugin for sieve, which can sort and filter mails depending on many different conditions, like presence and/or value of any header. You can deploy a single system wide default script for all users, which can be complemented or overwritten by individual users (depending on configuration). OK, so a delivery-time sieve. See http://wiki.dovecot.org/LDA/Sieve If you use dovecot 1.2, using the new Dovecot Sieve plugin is highly recommended above the old CMUsieve. See http://wiki.dovecot.org/LDA/Sieve/Dovecot Ubuntu has me at Dovecot version 1.1.11. Ubuntu has turned out to be difficult for managing upgrades when packages it has are instead compiled from source. So being up to date at the source level is going to have to wait until I get some Slackware running here (originally Ubuntu was chosen because of the wider variety of ready-to-install packages). The really recommended way nowadays is different though: *reject* as much spam as possible in the MTA in the receiving SMTP dialog, with a combination of blacklists, sender validation (e.g.DKIM), potentially greylisting (as you deem appropriate) and lastly content scanning with s.th. like amavis+spamassassin. I'd not expect a noticable amount of false positives, and in the rare cases the sender will be notified. The false positives rate of users scanning through a folder of tagged mail will be higher. I've had periodic high FPs with content detection in the past (on other mail servers I didn't control). So greylisting for that is what I will likely be doing, at least at first. How much I shift to blacklisting will depend on the feedback from my own users (hey, my Junk-spam folder has too much spam or How can I find that one false positive if there are a million spams in there). Tagging ... yeah, clearly I would do that for anything not rejected at SMTP time. How to get Dovecot deliver to put those ... well, OK, I'll look into Sieve instead of writing my own shim. A howto-like page to describe the usage of some dovecot features in an anti- spam context wouldn't hurt, OTOH there is not much anti-spam specific stuff at all at the dovecot side. Unfortunately, a good document would likely have to be specific to the LDA, MTA, and spam tools. Given so many combinations, that's a lot documents, or a rather complex one. I can certainly use my notes from my own solutions and make something specific to Dovecot and Postfix and whatever spam tools I use (could be more than one).
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 11:01, William Blunn bill+dove...@blunn.org wrote: Get your anti-spam software to add header records to the messages, then use Dovecot LDA Sieve to filter based on those header records. So I can run Sieve as part of the deliver? Or does it run soon after that?
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 12:17, Stan Hoeppner s...@hardwarefreak.com wrote: Phil Howard put forth on 6/1/2010 9:15 AM: On Tue, Jun 1, 2010 at 10:04, Jerry dovecot.u...@seibercom.net wrote: The fight against SPAM is NOT Dovecots responsibility. Whose responsibility is it to get detected spam delivered into the correct folder? Dovecot deliver is typically used to deliver into the Inbox(es). It would be involved. It's the MTA's responsibility to _REJECT_ spam attempts at SMTP time. It's the mail OP's responsibility to properly configure the MTA to do so. Spam fighting should be performed pre queue, not post queue. I suggest you bone up on modern spam fighting techniques. Post queue content analysis is not the proper way to fight spam, especially in 2010. For my own personal server, I would agree. Even for a general email or freemail service I would agree. However, for certain business environments, even a few false positives can be very troubling to management. Greylisting, and with shades of grey, is often more appropriate. Given that this puts mail where it will likely never be read or responded to, certainly doing as much as you can at SMTP time for a 5XX rejects is preferred, so that legitimate mail that would otherwise not be responded to is known to be not delivered where it would be read. But this can't be for everything, as even a FP rate of 0.01% is too much. So in the end, there has to be an administrative policy decision as to what to do with what is detected as spam. And if I can find a way, I'd like to even set that up so end USER policy can be applied even at SMTP time (e.g. user decides if spam is blacklisted or greylisted, along with user specified whitelists). But that's pushing the boundaries for now. When I get time, I will look into that degree of control.
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 12:55, William Blunn bill+dove...@blunn.org wrote: The Dovecot Sieve plugin provides mail filtering facilities at time of final message delivery using the Sieve (RFC 5228 http://www.ietf.org/rfc/rfc5228.txt) language. Sieve is implemented as a plugin to deliver http://wiki.dovecot.org/LDA. I hadn't gotten to reading much about Sieve, yet, as what I had read said it was something IMAP users would run. So I didn't realize (then) that I needed to read about it. It was in the list for stuff to read about, but just for later. It is now bumped up and I will be reading about it this week (the email server is a fraction of what is going on here).
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 12:41, Stan Hoeppner s...@hardwarefreak.com wrote: Law firm or heavily FED GOV regulated industry? Or just HUA (head up ass) management with zero understanding of email and spam? People dealing with a large variety of outside clients and potential clients where bouncing THEIR mail gives bad impressions in too many cases (already has with the outside mail provider we were using up until the move, which was expedited for just such issues). FYI, you forgot the marketing industry (not counting the bulk email marketing industry ... although we are not that). Also, lots of media and other industries can have this issue, as well as GOV itself.
Re: [Dovecot] Sieve GUI
On Tue, Jun 1, 2010 at 13:07, Gerhard Waldemair gerh...@waldemair.com wrote: Is there à GUI for sieve. I don't have installed Squrielmail, but won't like to manage the Rules per ssh I would hope there would be a way to put Sieve scripts in place via IMAP itself. Otherwise I won't be able to do that on my server (since there won't be system user accounts for mail users ... it's one system user virtual here).
Re: [Dovecot] Intermittent timeout issues
On Tue, Jun 1, 2010 at 12:58, Del Stoliker dstoli...@alphagraphics.com wrote: It almost sounds like the firewall is blocking external access except that: * The version of dovecot was the thing that changed * Everything starts working again after a dovecot restart It sounds like the firewall is doing something to the connections. I know that at least my IMAP client (Evolution) likes to hang on to an existing connection, even if nothing responds on the other end, for at least 15 minutes. By then the TCP layer drops it. For testing purposes, I've piped that connection coming from my desktop into a local port listened to by a program that relays all traffic to a connection it makes to the IMAP server. I can see it making requests and getting no answers if the server isn't responsive. If I kill that relay program, the client remakes the connection and all is well if a new connection can be established. I have seen a related issue with another service which was due to the firewall shifting the TCP sequence numbers for security purposes, combined with routing asymmetry where return packets bypassed the firewall sometimes (the packets are discarded, rather than exposing an easy TCP takedown by forgery). In your case, outside users are the ones involved, so this scenario would only apply for you if there is a way for outsiders to get around the firewall. Otherwise, your firewall may be losing connection state info and discarding further traffic without sending a TCP reset (usually they don't, but if you can configure it to do so, that can at least let new connections be made faster than the TCP stall timeout). Else, more memory or paid features or whatever for the firewall to hold more connection states. Or maybe it can be configured to pass all of this particular traffic without any TCP connection management features. Look at TCP packets on each side of the firewall for the IMAP service and see if the sequence numbers or even source port numbers are different.
Re: [Dovecot] Sieve GUI
On Tue, Jun 1, 2010 at 13:42, e-frog e-f...@gmx.de wrote: This might be helpful: http://wiki.dovecot.org/ManageSieve/ Right now only time for a quick look. Looks like yet another server with yet another protocol (via port 4190), rather than integrated into IMAP as an extension. I'm doubting all my email clients have support for this. Looks like I will need to do a lot of reading on Sieve and ManageSieve to be sure it's safe. For one thing, I want to turn the vacation feature off unless it can cross check a list of valid senders (user contacts).
Re: [Dovecot] Sieve GUI
On Tue, Jun 1, 2010 at 14:45, Frank Cusack frank+lists/dove...@linetwo.net wrote: On 6/1/10 2:10 PM -0400 Phil Howard wrote: On Tue, Jun 1, 2010 at 13:42, e-frog e-f...@gmx.de wrote: This might be helpful: http://wiki.dovecot.org/ManageSieve/ Right now only time for a quick look. Looks like yet another server with yet another protocol (via port 4190), rather than integrated into IMAP as an extension. I'm doubting all my email clients have support for this. You're wrong on it being another server; it does work within IMAP. But you are right in that almost no clients support it. Mulberry and thunderbird are the only ones I know of. However, there are web clients that work and given that sieve scripts are rarely modified it probably isn't too hard to justify forcing your user to go to a web page; much more possible than forcing them to change mail readers. The RFC describes otherwise ... e.g. a new protocol, which would connect to a new daemon. An extension to an Email client can still connect over to port 4190 to do the ManageSieve thing. Doing that via a web site, which can also support a few canned scripts or script options, probably is a better route. But our internal web server is still on the to-do list, so that's gonna wait, here. In the mean time, I just need a short term solution to divert tagged-as-spam messages into the INBOX-spam (or something like that) folder. I'll probably just go ahead and write a shim program in C (as easy for me as a script is for others) to check for the tag and add the -m option to deliver as needed. Looks like I will need to do a lot of reading on Sieve and ManageSieve to be sure it's safe. For one thing, I want to turn the vacation feature off unless it can cross check a list of valid senders (user contacts). Not possible. You'd have to implement your own customized support for that. I guess for now I'll be leaving this out. I don't want to be a vacation backscatter source.
Re: [Dovecot] Dovecot aspects of fighting spam
On Tue, Jun 1, 2010 at 14:56, Frank Cusack frank+lists/dove...@linetwo.net wrote: Man oh man. You don't have much experience with mail and it sounds like you are starting from scratch. You have your work cut out for you. :) Well, actually I do. A lot of it is way way back there with sendmail. Back then I coded sendmail.cf by hand and avoided m4. I have used Postfix previously, too, but in a very different circumstance, and even then it was several years back. The standard way to filter spam is [all of]: a) stop it at the border, then For my current situation, this will be a smaller amount than I've done in the past. b) content analysis and tagging, then I haven't done this in the past for my own mail servers primarily because of principle (e.g. to me UCE isn't about what it said, but whether it is unsolicited ... which indirectly means what is said ... and bulk ... in summary, it's about behaviour, not about the message). But in this new situation, I have to rebalance these ideas to achieve different goals. Being principled doesn't count in this situation. A potential client isn't going to be turned down or lectured just because the ISP they use is spammer friendly. I don't really like that, but that's the way it is. c) filtering at delivery time per-user filters for part (b) on the server side is a dead end IMHO but there are some programs that do this and dovecot can work with them (by having folders that train the filter). modern mail clients, however, have their own spam filters now, which are by definition personalized. anyway, part (a) and (b) have nothing to do with dovecot. the standard way to do part (c) is to have an X-Spam: header which a sieve script filters on. the -m flag to deliver is not really for spam filtering. Since sieve looks like it will be a problem right now, until I get a solution to that, I'm seriously considering this solution. A shim program I write in C will be run from Postfix master.cf just as Dovecot deliver is now. I'd basically change the executable path to the shim program. The shim program will read the message (I assume from stdin) up to 1MB or the end of headers. If the body isn't reached by 1MB it goes into the spam folder. If the X-Spam: header is present with a sufficient probability of spam, it goes into the spam folder. Else it goes into the INBOX. Set up a command argument list to run deliver, and include -m with the folder name if this goes to the spam folder. Set up pipes, fork, and child will exec deliver with that argument list. Pipe the buffer that was read in to deliver until it is empty, then pipe any remaining stdin to deliver all as one stream. Wait for deliver to exit and capture its exit status, and exit with the same status. Postfix should then know if delivery succeeded or failed. I may set up a web mail service later, and will see if I can do Sieve within that.
Re: [Dovecot] recipient_delimiter
On Fri, May 28, 2010 at 09:55, Charles Marcus cmar...@media-brokers.com wrote: On 2010-05-27 1:42 PM, Phil Howard wrote: Yup, there was a 2nd setting nearly at the bottom of the file, and it was different. Thanks for catching that. This is why you *always* go by what output pof postconf -n says, not what you think you put in main.cf. You wasted a lot of time (yours and others here) on that unnecessarily... My main.cf file has the comments (my own that explains why settings are there, not the default comments). It is the easier to read file. Even then, I was also reading the postconf -n output and just didn't see the subtle difference (I was thinking more along the lines of what else is needed). What might be needed to avoid things like this is something to compare them. Or maybe postfix giving an error when a conflicting setting is encountered (or is there some basis for always allowing settings that change previous settings).
Re: [Dovecot] recipient_delimiter
On Fri, May 28, 2010 at 12:11, Jerry dovecot.u...@seibercom.net wrote: On Fri, 28 May 2010 11:25:17 -0400 Phil Howard ttip...@gmail.com articulated: My main.cf file has the comments (my own that explains why settings are there, not the default comments). It is the easier to read file. Even then, I was also reading the postconf -n output and just didn't see the subtle difference (I was thinking more along the lines of what else is needed). What might be needed to avoid things like this is something to compare them. Or maybe postfix giving an error when a conflicting setting is encountered (or is there some basis for always allowing settings that change previous settings). In the Postfix main.cf file, the last entry wins. This is one of the reasons to use postconf -n and NOT what you think the correct entry is. The output of postconf -n shows what Postfix sees. To sum it up: Provide output from the postfinger tool. This can be found at http://ftp.wl0.org/SOURCES/postfinger. If the problem is SASL related, consider including the output from the saslfinger tool. This can be found at http://postfix.state-of-mind.de/patrick.koetter/saslfinger/. If the problem is about too much mail in the queue, consider including output from the qshape tool, as described in the QSHAPE_README file. If the problem is protocol related (connections time out, or an SMTP server complains about syntax errors etc.) consider recording a session with tcpdump, as described in the DEBUG_README document. Have you seen any config check tools?
Re: [Dovecot] purge maildir folder after sa-learn
On Wed, May 26, 2010 at 06:13, Timo Sirainen t...@iki.fi wrote: On Wed, 2010-05-26 at 09:56 +0200, Florian Effenberger wrote: rm -f /path/to/Maildir/.spam-learn/* Oh, I really ment .spam-learn/cur/* (and maybe .spam-learn/new/* if you make it learn from both and it doesn't move mails to cur/). Can I simply delete the files, don't I destroy any indices or slow down Dovecot operation because of index rebuilding? Keep Dovecot's index files around so that they won't get deleted while Dovecot is trying to read/write them. You won't slow down anything by manually deleting maildir files. So basically, just moving mail files out (to a learn queue or whatever), from either new or cur is safe for Dovecot? That would be simpler.
Re: [Dovecot] recipient_delimiter
On Fri, May 28, 2010 at 14:06, Charles Marcus cmar...@media-brokers.com wrote: On 2010-05-28 1:00 PM, Phil Howard wrote: If the problem is protocol related (connections time out, or an SMTP server complains about syntax errors etc.) consider recording a session with tcpdump, as described in the DEBUG_README document. Have you seen any config check tools? Yes - it's called a brain. ;) I think you are missing the point. A config check tool would be sifting through the details of the main.cf file, and the postconf-n.out file, and reporting the difference between what one thinks they have configured and what Postfix understands the configuration to be. Seriously - only you know your environment well enough to evaluate any given config. I think you are missing the point. Since what Postfix uses as the configuration isn't guaranteed to be what is coded ... because config items do get changed by subsequent config items ... a tool that can compare things comes in valuable. We all know that doing things in less time is a good thing (or else there would be not complaint about wasting time). But manually sifting through two files, point by point, and cross checking everything, every time, is as much, or more, of a time waster. And given that it would be done quite often, one is wasting a lot of time if they carry out that process. This tool would have to understand how Postfix interprets the config file (maybe just sufficient to know that conflicting config items don't produce errors or warnings in postfix or postconf), and just produce the warnings ... hey dude, you specified foo = 1 and later foo = 2 ... can't have it both ways, so you better go check on that. Use the tools, Luke... postconf -n and dovecot (or doveconf for 2.0) -n will complain for syntax errors, but it is up to you to evaluate the output. But it doesn't complain for conflict errors. And that was the class of error that happened.
Re: [Dovecot] recipient_delimiter
On Fri, May 28, 2010 at 15:28, Jerry dovecot.u...@seibercom.net wrote: Exactly what type of system are you trying to support? I fail to understand why you are constantly changing the base Postfix configuration. I add/delete users on a virtually daily basis, however, once my basic Postifx configuration was setup, I have had no reason to mess with it. If you are modifying it on a daily basis, you are probably doing something wrong. One that is just getting started. I expect to get to a point where I don't need to modify it. Not there, yet. We needed to get going on this sooner than originally planned, so not everything is in place, yet. For example, spam and virus trapping are still to do (next [*]). And tweaks are still possible for things like file locations as automation processes are added (later). I bet you have those thing already in place and thus don't need to change it. [*] Still trying to figure out how to make SpamAssassin put what it thinks is spam into an incoming spam box, rather than the usual INBOX (also seeking to find what name of such a box most people use). And that's in addition to another mailbox which users will move spam to, that comes into their INBOX, to be extracted at the server for training. This tool would have to understand how Postfix interprets the config file (maybe just sufficient to know that conflicting config items don't produce errors or warnings in postfix or postconf), and just produce the warnings ... hey dude, you specified foo = 1 and later foo = 2 ... can't have it both ways, so you better go check on that. Correct, you can only have the last one (Postfix). I am not sure about Dovecot. Then again, I don't enter rules haphazardly. Dovecot has sections and can do multiple sections. This is the Dovecot mailing list so you can ask about it here. Again, the output of postconf -n is what Postfix sees and understands. If you think you have duplicate values in the file, try 'grep' to locate them. It doesn't make any difference though since the last one wins no matter what. NOTE: One known exception is: dovecot_destination_recipient_limit = 1 This does not display with postconf -n. Wietse once explained why; however, I don't remember why any more. By the way, if you are doing virtual delivery with Postfix/Dovecot, you should probably have that in the main.cf file. As always, YMMV. I'm assuming because it is a dynamically named setting for a transport. Use the tools, Luke... postconf -n and dovecot (or doveconf for 2.0) -n will complain for syntax errors, but it is up to you to evaluate the output. But it doesn't complain for conflict errors. And that was the class of error that happened. What conflict error? It would not happen in Postfix's main.cf file. Two like settings, where one would not be used, regardless of which (whether it's a first setting wins or a last setting wins system), is a conflict in my book. I'd have made the system flag it as an error (except maybe if both settings would have the exact same effect) and maybe not even start up. If postconf -n carried comments along, then it could be used as a config linter ... let it's output replace the original and that will be the one to edit for the next change.
Re: [Dovecot] recipient_delimiter
On Fri, May 28, 2010 at 16:53, Charles Marcus cmar...@media-brokers.com wrote: On 2010-05-28 4:49 PM, Phil Howard wrote: If postconf -n carried comments along, then it could be used as a config linter ... let it's output replace the original and that will be the one to edit for the next change. The whole purpose of the -n output is to provide clean, easy to read *settings* as seen by postfix (as opposed to comments that are meant for people). So you are saying that this is not meant for people?
Re: [Dovecot] recipient_delimiter
On Wed, May 26, 2010 at 16:59, Jerry dovecot.u...@seibercom.net wrote: On or about line 821 in the Dovecot.conf file: #auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ If you have not all ready done so, add the + to the end of the list and un-comment the line. I believe you will have to restart 'dovecot' for the change to take affect. Now added. By the way, Postfix must be properly configured to pass the email address to dovecot. I leave that as an exercise to the user. You might want to start here thought: http://wiki.dovecot.org/LDA/Postfix The question I have next is, what is properly. Should Postfix pass phil+how...@example.com or p...@example.com ... and in which place (on the command line to deliver? ... the first line passed through the pipe? ... in the RFC822 To: header?). How does Dovecot know I have recipient_delimiter = + (vs. recipient_delimiter = - or such) in main.cf, or should it matter? It would seem to me that if Dovecot has to figure out that phil+how...@example.com goes into the mailbox for p...@example.com then it needs to know that + is the character being used for that. Otherwise it seems that Postfix should tell it that the delivery is to p...@example.com. One frequent confuse I see in referencing documentation for a problem, is understanding what it is that needs to be achieved, in order for the proper documented configuration to be determined.
Re: [Dovecot] recipient_delimiter
On Thu, May 27, 2010 at 10:50, Gregory Finch gfi...@ldmltd.ca wrote:
I've never had to touch auth_username_chars to get this to work.
In main.cf, I've set:
mailbox_transport = dovecot
I have virtual_transport = dovecot
recipient_delimiter = +
I have that now (didn't when I initially set up the server assuming it
was a default, but later added it)
And in master.cf:
# Dovecot
dovecot unix - n n - - pipe
flags=DR user=vmail:vmail
argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user} -a
${recipient} -n -m ${extension}
Mine is a bit different:
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f
${sender} -d ${us...@${nexthop}
I'm still not sure what nexthop means but it seems to make it work.
I may not need all the options to deliver, but this does work nice.
One thing I did notice is that I needed to add the %L arg in my passdb
and userdb definitions in my dovecot.conf in order to make the mailbox
name case insensitive:
passdb:
driver: passwd-file
args: username_format=%Ln /etc/dovecot/passwd
userdb:
driver: static
args: uid=500 gid=500 home=/srv/dovecot/%Ln
# 1.1.11: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.31-19-server x86_64 Ubuntu 9.10 ext3
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot/error.log
info_log_path: /var/log/dovecot/info.log
log_timestamp: %Y-%m-%d %H:%M:%S
protocols: imap pop3 imaps pop3s
listen: 172.30.0.24, [fc00::18], 127.0.0.1, [::1]
ssl_cert_file: /etc/ssl/certs/imap.example.com.crt
ssl_key_file: /etc/ssl/private/imap.example.com.key
ssl_parameters_regenerate: 24
ssl_cipher_list:
ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM
login_dir: /var/run/dovecot//login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(pop3): no
mail_max_userip_connections(default): 10
mail_max_userip_connections(imap): 10
mail_max_userip_connections(pop3): 3
verbose_proctitle: yes
first_valid_uid: 250
mail_privileged_group: mail
mail_uid: vmail
mail_gid: vmail
mail_location: maildir:/home/mail/%Ld/%Ln/mail
mail_debug: yes
mail_executable(default): /usr/lib/dovecot/imap
mail_executable(imap): /usr/lib/dovecot/imap
mail_executable(pop3): /usr/lib/dovecot/pop3
mail_process_size: 768
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(pop3):
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
auth default:
mechanisms: plain login
username_chars:
abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz01234567890...@+
username_format: %...@%ld
verbose: yes
debug: yes
debug_passwords: yes
passdb:
driver: passwd-file
args: username_format=...@%ld /etc/mailauth/deny
deny: yes
passdb:
driver: passwd-file
args: username_format=%Ln /etc/mailauth/%Ld/deny
deny: yes
passdb:
driver: passwd-file
args: scheme=crypt username_format=...@%ld /etc/mailauth/passwd
passdb:
driver: passwd-file
args: scheme=crypt username_format=%Ln /etc/mailauth/%Ld/passwd
userdb:
driver: passwd-file
args: username_format=...@%ld /etc/mailauth/passwd
userdb:
driver: passwd-file
args: username_format=%Ln /etc/mailauth/%Ld/passwd
socket:
type: listen
client:
path: /var/spool/postfix/private/dovecot-auth
mode: 432
user: postfix
group: postfix
master:
path: /var/run/dovecot/auth-master
mode: 384
user: vmail
group: vmail
alias_database = cdb:/etc/postfix/aliases
alias_maps = cdb:/etc/aliases
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/lib/postfix
data_directory = /var/lib/postfix
default_destination_concurrency_limit = 2
default_privs = nobody
in_flow_delay = 1s
inet_protocols = ipv4, ipv6
local_destination_concurrency_limit = 2
mail_owner = postfix
mydomain = example.com
myhostname = mx.example.com
mynetworks = 127.0.0.0/8, 172.16.0.0/16, 172.20.0.0/16, 172.30.0.0/16,
[::1]/128, [fc00::]/48, [fe80::]/48
myorigin = $mydomain
proxy_interfaces = MY OUTSIDE IP
queue_directory = /var/spool/postfix
recipient_delimiter = -
smtp_bind_address = 172.30.0.25
smtp_bind_address6 = fc00::25
smtpd_banner = $myhostname ESMTP - UNSOLICITED BULK OR ILLEGAL
Re: [Dovecot] recipient_delimiter
On Wed, May 26, 2010 at 16:59, Jerry dovecot.u...@seibercom.net wrote: On or about line 821 in the Dovecot.conf file: Actually, a lot earlier in my trimmed down version :-) #auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ If you have not all ready done so, add the + to the end of the list and un-comment the line. I believe you will have to restart 'dovecot' for the change to take affect. This alone made it work. Thanks! Given Gregory Finch's response, I'll have to guess that the default included + or something else is going on. I did have auth_username_chars uncommented (so I was not using the default) and without the + earlier.
Re: [Dovecot] symlinking maildirs
On Thu, May 27, 2010 at 01:52, Pascal Volk user+dove...@localhost.localdomain.org wrote: On 05/26/2010 10:27 PM Phil Howard wrote: [snipped X-posting] See this Thread: http://marc.info/?l=postfix-usersm=127490477231031w=2 The symlinking did work, though. That was with userids added to auth, too. So dovecot deliver thought it was delivering to the other user, but the effect was the intended user.
Re: [Dovecot] recipient_delimiter
On Thu, May 27, 2010 at 11:34, Phil Howard ttip...@gmail.com wrote: On Wed, May 26, 2010 at 16:59, Jerry dovecot.u...@seibercom.net wrote: On or about line 821 in the Dovecot.conf file: Actually, a lot earlier in my trimmed down version :-) #auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@ If you have not all ready done so, add the + to the end of the list and un-comment the line. I believe you will have to restart 'dovecot' for the change to take affect. This alone made it work. Thanks! Given Gregory Finch's response, I'll have to guess that the default included + or something else is going on. I did have auth_username_chars uncommented (so I was not using the default) and without the + earlier. I spoke too quick. It seems to be intermittent. May 27 11:00:02 eth0 postfix/pipe[25377]: DCA7868417: to=phil+how...@example.com, relay=dovecot, delay=0.27, delays=0.21/0.01/0/0.05, dsn=4.1.1, status=SOFTBOUNCE (user unknown) May 27 11:04:27 eth0 postfix/pipe[25411]: 16DD168411: to=phil+how...@example.com, relay=dovecot, delay=397, delays=397/0.01/0/0.04, dsn=4.1.1, status=SOFTBOUNCE (user unknown) May 27 11:04:27 eth0 postfix/pipe[25413]: 864CB6846F: to=phil+how...@example.com, relay=dovecot, delay=67732, delays=67732/0.01/0/0.05, dsn=4.1.1, status=SOFTBOUNCE (user unknown) May 27 11:09:09 eth0 postfix/pipe[25605]: DCA7868417: to=phil+how...@example.com, relay=dovecot, delay=547, delays=547/0.01/0/0.08, dsn=2.0.0, status=sent (delivered via dovecot service) May 27 11:14:08 eth0 postfix/pipe[25682]: 16DD168411: to=phil+how...@example.com, relay=dovecot, delay=978, delays=977/0.01/0/0.09, dsn=2.0.0, status=sent (delivered via dovecot service) May 27 11:46:11 eth0 postfix/pipe[26384]: DEC3368411: to=phil+how...@example.com, relay=dovecot, delay=13, delays=13/0/0/0.04, dsn=4.1.1, status=SOFTBOUNCE (user unknown) May 27 11:46:54 eth0 postfix/pipe[26384]: DA42468412: to=phil+how...@example.com, relay=dovecot, delay=13, delays=13/0/0/0.04, dsn=4.1.1, status=SOFTBOUNCE (user unknown) May 27 11:47:37 eth0 postfix/pipe[26384]: E612968413: to=phil+how...@example.com, relay=dovecot, delay=13, delays=13/0/0/0.03, dsn=4.1.1, status=SOFTBOUNCE (user unknown)
Re: [Dovecot] recipient_delimiter
On Wed, May 26, 2010 at 16:59, Jerry dovecot.u...@seibercom.net wrote:
By the way, Postfix must be properly configured to pass the email
address to dovecot. I leave that as an exercise to the user. You might
want to start here thought:
http://wiki.dovecot.org/LDA/Postfix
I already have this:
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f
${sender} -d ${us...@${nexthop}
Maybe /usr/lib/dovecot/deliver is just the path the Ubuntu package
team chose for Dovecot?
The intent is to deliver into the mailbox INBOX for the user without
the extension, but with the extension still in there where user
scripts can see it and choose to act on it. I don't want an explosion
of folders in the event of a DoS attack via massively randomly
generated extensions.
So the big question is, is Postfix supposed to split at + and pass
user instead of user+ext to the transport program, or is
dovecot/deliver supposed to split at + and and use that for %n ? If I
knew what was SUPPOSED to happen, it would be easier to make use of
understanding some of the logic to make it happen.
It's one thing to know what something MEANS. It's entirely different
to know what something DOES. I think the latter is more meaningful,
but it also depends highly on knowing that aspect of everything.
Re: [Dovecot] recipient_delimiter
On Thu, May 27, 2010 at 12:52, Gregory Finch gfi...@ldmltd.ca wrote:
On 2010-05-27 8:27 AM, Phil Howard wrote:
On Thu, May 27, 2010 at 10:50, Gregory Finch gfi...@ldmltd.ca wrote:
I've never had to touch auth_username_chars to get this to work.
In main.cf, I've set:
mailbox_transport = dovecot
I have virtual_transport = dovecot
recipient_delimiter = +
I have that now (didn't when I initially set up the server assuming it
was a default, but later added it)
And in master.cf:
# Dovecot
dovecot unix - n n - - pipe
flags=DR user=vmail:vmail
argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user} -a
${recipient} -n -m ${extension}
Mine is a bit different:
dovecot unix - n n - - pipe
flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f
${sender} -d ${us...@${nexthop}
I'm still not sure what nexthop means but it seems to make it work.
I may not need all the options to deliver, but this does work nice.
One thing I did notice is that I needed to add the %L arg in my passdb
and userdb definitions in my dovecot.conf in order to make the mailbox
name case insensitive:
passdb:
driver: passwd-file
args: username_format=%Ln /etc/dovecot/passwd
userdb:
driver: static
args: uid=500 gid=500 home=/srv/dovecot/%Ln
# 1.1.11: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.31-19-server x86_64 Ubuntu 9.10 ext3
base_dir: /var/run/dovecot/
log_path: /var/log/dovecot/error.log
info_log_path: /var/log/dovecot/info.log
log_timestamp: %Y-%m-%d %H:%M:%S
protocols: imap pop3 imaps pop3s
listen: 172.30.0.24, [fc00::18], 127.0.0.1, [::1]
ssl_cert_file: /etc/ssl/certs/imap.example.com.crt
ssl_key_file: /etc/ssl/private/imap.example.com.key
ssl_parameters_regenerate: 24
ssl_cipher_list:
ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM
login_dir: /var/run/dovecot//login
login_executable(default): /usr/lib/dovecot/imap-login
login_executable(imap): /usr/lib/dovecot/imap-login
login_executable(pop3): /usr/lib/dovecot/pop3-login
login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly
prohibited
login_greeting_capability(default): yes
login_greeting_capability(imap): yes
login_greeting_capability(pop3): no
mail_max_userip_connections(default): 10
mail_max_userip_connections(imap): 10
mail_max_userip_connections(pop3): 3
verbose_proctitle: yes
first_valid_uid: 250
mail_privileged_group: mail
mail_uid: vmail
mail_gid: vmail
mail_location: maildir:/home/mail/%Ld/%Ln/mail
mail_debug: yes
mail_executable(default): /usr/lib/dovecot/imap
mail_executable(imap): /usr/lib/dovecot/imap
mail_executable(pop3): /usr/lib/dovecot/pop3
mail_process_size: 768
mail_plugin_dir(default): /usr/lib/dovecot/modules/imap
mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap
mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3
imap_client_workarounds(default): outlook-idle delay-newmail
imap_client_workarounds(imap): outlook-idle delay-newmail
imap_client_workarounds(pop3):
pop3_client_workarounds(default):
pop3_client_workarounds(imap):
pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh
auth default:
mechanisms: plain login
username_chars:
abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz01234567890...@+
username_format: %...@%ld
verbose: yes
debug: yes
debug_passwords: yes
passdb:
driver: passwd-file
args: username_format=...@%ld /etc/mailauth/deny
deny: yes
passdb:
driver: passwd-file
args: username_format=%Ln /etc/mailauth/%Ld/deny
deny: yes
passdb:
driver: passwd-file
args: scheme=crypt username_format=...@%ld /etc/mailauth/passwd
passdb:
driver: passwd-file
args: scheme=crypt username_format=%Ln /etc/mailauth/%Ld/passwd
userdb:
driver: passwd-file
args: username_format=...@%ld /etc/mailauth/passwd
userdb:
driver: passwd-file
args: username_format=%Ln /etc/mailauth/%Ld/passwd
socket:
type: listen
client:
path: /var/spool/postfix/private/dovecot-auth
mode: 432
user: postfix
group: postfix
master:
path: /var/run/dovecot/auth-master
mode: 384
user: vmail
group: vmail
alias_database = cdb:/etc/postfix/aliases
alias_maps = cdb:/etc/aliases
command_directory = /usr/sbin
config_directory = /etc/postfix
daemon_directory = /usr/lib/postfix
data_directory = /var/lib/postfix
default_destination_concurrency_limit = 2
default_privs = nobody
in_flow_delay = 1s
inet_protocols = ipv4, ipv6
local_destination_concurrency_limit = 2
mail_owner = postfix
mydomain = example.com
myhostname = mx.example.com
mynetworks = 127.0.0.0/8, 172.16.0.0/16, 172.20.0.0/16, 172.30.0.0/16,
[::1]/128, [fc00::]/48, [fe80::]/48
myorigin
Re: [Dovecot] recipient_delimiter
On Thu, May 27, 2010 at 12:52, Gregory Finch gfi...@ldmltd.ca wrote: recipient_delimiter = - ^^ do you mean to have a + here instead? Yup, there was a 2nd setting nearly at the bottom of the file, and it was different. Thanks for catching that.
Re: [Dovecot] STARTTLS does not seem to work
On Tue, May 25, 2010 at 16:31, Jerrale Gayle jerralega...@sheltoncomputers.com wrote: The openssl client will connect you in plain text to your imap server where you can manually do login (AUTH LOGIN) and browse through your imap folders just like you use your SSH shell. This is a sufficient enough test. Refer here, after doing what Mike Abbott told you to do with openssl s_client: http://www.macgeekery.com/tips/troubleshooting/troubleshooting_imap The test I want to do requires deleting the existing mail after fetching it ... so that subsequent runs won't see that mail, again. There will be timestamp coded info in that email, too (though it would fit on the subject). So it seems something like fetchmail could work. But I do need to do this test in a variety of different ways specified by a script, so the ability to specify those things on the command line or environment is preferred over doing it by means of a config file. Specifying them by means of running an interactive program is out. These things include what server to connect to (by hostname or IP address), what certificate hostname to expect (may be different than the connect host, since these tests may be running through tunnels), what port to connect to, whether to operate in clear vs. use STARTTLS vs. use SSL wrapper, what password authentication scheme to use, the password itself, and the local directory for the mail transfer (where to get mail to send/submit, or where to deposit mail fetched ... maildir format preferred for that). This is about operational testing, not implementation or deployment testing. It will be run on a regular basis and the scripts will log the results in various places, including notifying operators and/or administrators depending on the issues discovered. Among the tests will include tests that are expected to fail (for example connecting to port 465, or logging in without enabling TLS) and will raise an issue if they succeed. Every test unit will have a unique u...@domain. Some tests will even be specifically testing domains (every domain, though not every hostname, will have a test unit). I expect a few hundred test units to be deployed for the mail system (some offsite ... that's to be tested, too).
[Dovecot] warning: pipe flag `D' requires dovecot_destination_recipient_limit = 1
I'm getting this message, which looks like Postfix is the one logging it, but I don't know where it comes from or which configuration it needs to be placed in. Anyone know what this means? May 26 13:13:28 eth0 postfix/pipe[13821]: warning: pipe flag `D' requires dovecot_destination_recipient_limit = 1
Re: [Dovecot] warning: pipe flag `D' requires dovecot_destination_recipient_limit = 1
On Wed, May 26, 2010 at 13:35, Pascal Volk user+dove...@localhost.localdomain.org wrote: On 05/26/2010 07:19 PM Phil Howard wrote: I'm getting this message, which looks like Postfix is the one logging it, but I don't know where it comes from or which configuration it needs to be placed in. Anyone know what this means? Put it in `postconf -h config_directory`/main.cf. Also have a look into: man 5 postconf | less +/^transport_destination_recipient_limit Ah, tricky documentation. That explains why search didn't come up with it. I think they need a meta page for all the transports for search to find it.
[Dovecot] recipient_delimiter
I have recipient_delimiter = + in main.cf (Postfix). But deliveries to dovecot/deliver are still being rejected. It appears that the full email is still in the envelope (e.g. phil+how...@example.com which I want to be delivered to p...@example.com). Is that how it's supposed to work and Dovecot just doesn't support this?
[Dovecot] symlinking maildirs
I have some emails that I think are in the Postfix delivery queue going to Dovecot, but didn't get mapped to their alias (because I didn't have all the alias entries in when they were queued). I'm wondering if this trick would work. I have the mail being delivered to /home/mail/%Ld/%Ln/mail and am wondering if I could make a symlink at /home/mail/example.com/foo - bar and add a dummy password for f...@example.com and have that mail trying to go to f...@example.com really go to b...@example and make a successful delivery (it is soft bouncing right now).
Re: [Dovecot] dovecot/deliver ... Can't open log file /var/log/dovecot/error.log: Permission denied
On Fri, May 21, 2010 at 11:17, Charles Marcus cmar...@media-brokers.com wrote: On 2010-05-21 9:04 AM, Phil Howard wrote: Where is the reply to list button on Evolution? I don't see one there, either. All it has are reply and reply to all. The reply sends to the sender alone if it's a case where there are 2 addresses to send to (reply to all would send to 2 in that case). Thunderbird has it now (yay!), although you have to manually add the button to the toolbar. Keybd equiv is CTRL-SHIFT-L... Before that I always had to Reply To All then manually delete the direct/individual address... If you run across any Evolution developers ... at least before I convert to Thunderbird (can't do that right now, but that is an option once mail server issues are done) ... be sure to poke them about this.
[Dovecot] STARTTLS does not seem to work
I believe I have the configuration set to use START TLS on IMAP4 (143) and POP3 (110) ports. However, it does not seem to be working. Yet STARTTLS is listed as a capability (which tells me I probably do have it configured right). In the session below, 172.30.0.24 is the mail server I'm putting up. 64.26.60.229 is an outside mail service. A similar thing happens on POP3. The always-SSL/TLS ports (993 and 995) are working. There's very little documentation matching starttls. altair/phil /home/phil 162 telnet 172.30.0.24 143 Trying 172.30.0.24... Connected to 172.30.0.24. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED I18NLEVEL=1 STARTTLS LOGINDISABLED] AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited STARTTLS STARTTLS BAD Error in IMAP command received by server. ^]quit telnet quit Connection closed. altair/phil /home/phil 163 telnet 64.26.60.229 143 Trying 64.26.60.229... Connected to 64.26.60.229. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE STARTTLS] Courier-IMAP ready. Copyright 1998-2005 Double Precision, Inc. See COPYING for distribution information. STARTTLS STARTTLS OK Begin SSL/TLS negotiation now. ^]quit telnet quit Connection closed. altair/phil /home/phil 164 I do have disable_plaintext_auth = yes in my config file even though dovecot -n does not show it ... must be a default. # 1.1.11: /etc/dovecot/dovecot.conf # OS: Linux 2.6.31-19-server x86_64 Ubuntu 9.10 ext3 base_dir: /var/run/dovecot/ log_path: /var/log/dovecot/error.log info_log_path: /var/log/dovecot/info.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap pop3 imaps pop3s listen: 172.30.0.24, [fc00::18], 127.0.0.1, [::1] ssl_cert_file: /etc/ssl/certs/ssl-mail.pem ssl_key_file: /etc/ssl/private/ssl-mail.key ssl_parameters_regenerate: 24 ssl_cipher_list: ALL:!LOW:!SSLv2:ALL:!aNULL:!ADH:!eNULL:!EXP:RC4+RSA:+HIGH:+MEDIUM login_dir: /var/run/dovecot//login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_greeting: AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no mail_max_userip_connections(default): 10 mail_max_userip_connections(imap): 10 mail_max_userip_connections(pop3): 3 verbose_proctitle: yes first_valid_uid: 250 mail_privileged_group: mail mail_uid: vmail mail_gid: vmail mail_location: maildir:/home/mail/%Ld/%Ln/mail mail_debug: yes mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_process_size: 768 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 imap_client_workarounds(default): outlook-idle delay-newmail imap_client_workarounds(imap): outlook-idle delay-newmail imap_client_workarounds(pop3): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh auth default: mechanisms: plain login username_format: %...@%ld verbose: yes debug: yes debug_passwords: yes passdb: driver: passwd-file args: username_format=...@%ld /etc/mailauth/deny deny: yes passdb: driver: passwd-file args: username_format=%Ln /etc/mailauth/%Ld/deny deny: yes passdb: driver: passwd-file args: scheme=crypt username_format=...@%ld /etc/mailauth/passwd passdb: driver: passwd-file args: scheme=crypt username_format=%Ln /etc/mailauth/%Ld/passwd userdb: driver: passwd-file args: username_format=...@%ld /etc/mailauth/passwd userdb: driver: passwd-file args: username_format=%Ln /etc/mailauth/%Ld/passwd socket: type: listen client: path: /var/spool/postfix/private/dovecot-auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail
Re: [Dovecot] STARTTLS does not seem to work
On Mon, May 24, 2010 at 11:31, Mike Abbott michael.abb...@apple.com wrote: altair/phil /home/phil 162 telnet 172.30.0.24 143 Trying 172.30.0.24... Connected to 172.30.0.24. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED I18NLEVEL=1 STARTTLS LOGINDISABLED] AUTHORIZED USERS ONLY -- unauthorized access strictly prohibited STARTTLS STARTTLS BAD Error in IMAP command received by server. ^]quit Every IMAP command needs a command tag. Instead of STARTTLS try A STARTTLS. Interesting. So the outside service's software has a hack in there to accept it without the tag? I wonder why? Maybe it's a workaround because some client software does it that way. Anyway, with the tag it does work on IMAP. But it still fails on POP (which doesn't have tags, though I tried one anyway for kicks).
Re: [Dovecot] STARTTLS does not seem to work
On Mon, May 24, 2010 at 11:49, Mike Abbott michael.abb...@apple.com wrote: Anyway, with the tag it does work on IMAP. But it still fails on POP For POP3 the command is STLS. Well, that kinda complicates a STARTTLS tunnel :-) ... I was thinking of trying to do that to address some issues. OK, well, put the emphasis on seem in my subject. It seems I made assumptions about the protocols.
Re: [Dovecot] STARTTLS does not seem to work
On Mon, May 24, 2010 at 17:31, Mike Abbott michael.abb...@apple.com wrote: Well, that kinda complicates a STARTTLS tunnel Perhaps you might be interested in these commands. I'm not sure about their portability but they work tolerably well in scripts on Mac OS X 10.6. $ openssl s_client -connect yourhost:imap -starttls imap $ openssl s_client -connect yourhost:pop3 -starttls pop3 $ openssl s_client -connect yourhost:imaps $ openssl s_client -connect yourhost:pop3s $ openssl s_client -connect yourhost:smtp -starttls smtp Yeah, that can be used, perhaps best with expect or pexpect. I'm hoping to find tools that can do basic email functions at a higher level, where the user of the tool does not need to know the protocol details, but only needs to consider the same kinds of configuration aspects that configuring a regular email client involves (except without all the misleading and often erroneous terminology used by these GUI client developers ... such as TLS for STARTTLS/STLS on clear ports, and SSL for wrapped/tunneled TLS/SSL connections on always-encrypted ports as used in Evolution). At some point I think I need to learn the OpenSSL library API for C so I can write some command line tool apps of my own with it (now we're getting well off the Dovecot topic).
Re: [Dovecot] dovecot/deliver ... Can't open log file /var/log/dovecot/error.log: Permission denied
On Thu, May 20, 2010 at 18:45, Noel Butler noel.but...@ausics.net wrote: like you said, you dont really do mail servers Id LOVE to see you try even 100K users in mail server situation that is ever changing, you'd soon open your eyes up. I know it would mean more hits to the DB. The reason we moved from qmail/vpopmail CDB to qmail/vpopmail/mysql was for a MASSIVE IMPROVEMENT in performance, then added dovecot in for even more performance improvements, I too was hesitant, but a large university having similar problems to us made the change and it was like comparing a snail V porsche, I made our change based on their results and never looked back, of course we then had the sense to migrate to postfix and remove vpopmail from the equation altogether. Best move ever, so yes experience counts. CDB can still run circles around any *SQL DB. Sounds to me like you were using CDB wrong or other factors in your situation made CDB impractical. One such situation could be frequent updates. If you need to do frequent updates, and with 100K users that may well be the case, then CDB can be a loss. Did you try Berkeley DB? Now show me how this means CDB is bad for lookups. (please use reply to list, not reply to all) No such button. That's one of the reasons why mailing lists are lousy. Oh, since this is a list about an aspect of mail servers, I suppose it seems natural to communicate over a mailing list. OTOH, some people might need to communicate when mail isn't working. That's one of the reasons I acquired a Gmail account for this and Postfix subscription. So do you know a freemail service where there is a reply to list button? well I dont need two copies, and often yours get here first and accepted, therefor the list copy is discarded as duplicate, I prefer my lsit mail to be sorted by evolution into its respective mail folder. The fact gmail dont offer this is no excuse most other gmail users dont have this problem, its just another mess gmail creates, like their hopeless quoter segment handling, but , if you use a service you dont pay for then you cant bitch, but often because some dont know how to cut quoting, it ends up being 15 pages long and you have NFI who said what. Where is the reply to list button on Evolution? I don't see one there, either. All it has are reply and reply to all. The reply sends to the sender alone if it's a case where there are 2 addresses to send to (reply to all would send to 2 in that case). There are a number of posters on the list where the reply goes directly to the list alone. I don't know what it is they do with the headers to get it to come out that way. Maybe you can ask them what they do, then do that. I have a gmail a/c only for testing when someone whinges the cant get mail from them, i'd never rely on them for day to day communications, those that do, need to stop being lazy and make the extra effort, Evolution (my version anyway) has no short cut button., I have to hit the key combo manually, its not killing me to do so. I don't know what key you are talking about. anyway, I guess you'll think your way, and I'll think mine, going to be pointless continuing this thread it seems, since your not by your own admission a mail admin and obviously have not had to deal with the situations we have. It's not about thinking ... it's about seeing. I see faster lookup performance from CDB and similar technology than from MySQL or PostgreSQL. I haven't tried Oracle (nor is that ever likely to happen). I have tried SyBase and Ingres, and they were both quite bad. In the Sybase case, updates were fortunately only daily, so I could literally run a cron job to download all the records at night, and build a CDB-like DB, and have the lookups be done from there. With 34 million records, the download took about 4 hours. Lookups to Sybase took about 20 seconds each (and it was an indexed table). Lookups on the DB file were a tiny fraction of a second. I see problems with big database engines all the time. Sure, if you are running a big massive mail server with lots of updates, and SQL DB might well be the only choice. Tell me what DB GMAIL uses. Threads like this are one of the reasons I'm posting from GMAIL.
Re: [Dovecot] dovecot/deliver ... Can't open log file /var/log/dovecot/error.log: Permission denied
On Mon, May 17, 2010 at 17:14, Noel Butler noel.but...@ausics.net wrote: I guess you've never used it with tens and tens of thousands of users, let alone user numbers well into 6 figures and why on gods (or any) earth would I use that load of crap being backed up by another form? that clearly makes no sense, we have backup provisions being mysql replications sure but thats nothing like what you do. your method is pure insanity in this day and age. I've used a like technique with over 45 million records present. Was extremely fast. Beat the pants off SQL for the kinds of things this is good for, which is: ... simple key:value lookups I guess you've been bitten by a proper database solution given your apprehension for using one. It's called experience. I could explain many cases where SQL is overkill and overhead. But I don't do mail servers very much, so it would all be off-topic for this list. This is not the SQL/NOSQL battle zone. yes it is, if you only have a small number of users. Why would it be any slower if the 45 million records represented users instead of document IDs? (please use reply to list, not reply to all) No such button. That's one of the reasons why mailing lists are lousy. Oh, since this is a list about an aspect of mail servers, I suppose it seems natural to communicate over a mailing list. OTOH, some people might need to communicate when mail isn't working. That's one of the reasons I acquired a Gmail account for this and Postfix subscription. So do you know a freemail service where there is a reply to list button?
Re: [Dovecot] dovecot/deliver ... Can't open log file /var/log/dovecot/error.log: Permission denied
On Tue, May 18, 2010 at 01:46, Eray Aslan eray.as...@caf.com.tr wrote: We use Michael Tokarev's tinycdb: * stable on disk format * has atomic updates * has a tiny library in case you want to link it statically * uses a lot less memory when the process count is large, i.e. scales well * rebuilding the database a few times a day is just fine with a few million records. If you go into tens or hundreds of millions, test rebuild times before putting into production Using cdb is by no means a MUST but don't dissmiss it out of hand. It has its uses. I assume the update is rebuilding the CDB file from the data source, and when complete and tested (look up at least the first and last records added), doing a move to replace the old CDB so when the program using it does open, again (something it should do periodically and when it gets a SIGHUP), it gets the new one. Or has tinycdb figured a way to update in place? If I needed to change records fast, and have each one be active, fast, I'd certainly choose something other than CDB or equivalents to it. That might be Berkeley DB (BTDT). That might even be a big directory of millions of files on a B-tree based filesystem (BTDT). That might even be SQL (BTDT with 3 different SQL RDB systems). I pick what's right for the given circumstance (and just being mail does not narrow down the circumstance). I don't just use something because everyone else does. CDB fits a lot of circumstances well ... not all, fo sure ... but a lot (including my current case, so I wish it were available).
Re: [Dovecot] looking for IMAP testing tool
On Wed, May 19, 2010 at 05:24, Brian Candler b.cand...@pobox.com wrote: Then you're testing the whole environment: you'll need to deliver mail either by making SMTP connections or by invoking your LDA (e.g. sendmail) and piping the mail in - with some way of forcing it to look spammy or not spammy - to check the blocking. Then you'll use IMAP to retrieve them. This is clearly more than testing just IMAP; rather, you're testing the whole mail server platform and its configs. Yes, that is what I want to be testing. So I need a way to send mail via SMTP (including TLS and login authentication) as well as picking it up via IMAP. But I chose to only ask for the IMAP piece of it here (and the SMTP piece of it on the Postfix mailing list ... to which Wietse suggested expect and openssl s_client which I think I can handle using pexpect in Python). I know enough SMTP to do that end of things. I don't know the IMAP protocol at all, so something already built would help. But I only need a subset which is to just pick up all email from the inbox of a given u...@domain, deleting them from the server while depositing them in a designated directory. Each different class of test would be using a different mailbox. Some tests will, when things work as expected, get no mail at all (basically a test for rejecting mail). I don't know of any ready-made test framework for what you want, and I suspect it would end up looking much like a programming language by the time you were able to configure all the different tests for processing different flavours of incoming mail. An IMAP library might be doable (though not in Perl since I don't know that language and don't have the time to learn it), but the basic just pick up and delete all mail would be sufficient.
Re: [Dovecot] looking for IMAP testing tool
On Wed, May 19, 2010 at 07:52, William Blunn b...@blunn.org wrote: It sounds like you want a sort of toolbox of ready-made and tested components, such as an IMAP client, but with rich programmatic interfaces so that you only need to write a little bit of glue code to make it do exactly what you want. I don't really need a lot. I just need to pick up all mail in the inbox of a given u...@domain I login with, deposit each in a given directory, and delete them from the server. The only options I think I need are: what u...@domain to login as, whether to use clear, STARTTLS, or a basic TLS/SSL protocol, and what port number (143 or 993). I don't have any interest in other folders. I just want to see if mail was delivered and do so from a separate machine than the mail server itself (or see if IMAP is even up).
Re: [Dovecot] looking for IMAP testing tool
On Wed, May 19, 2010 at 10:04, Brian Candler b.cand...@pobox.com wrote: If you can do SMTP, you can do IMAP. This should get you started: a login f...@bar.com xyzzy a select inbox -- or a examine inbox for read-only a fetch 1:15 (rfc822) a store 1:15 +flags (\Deleted) a expunge a logout Also useful: a namespace-- folder separator chars a list * -- list folder hierarchy Flick through RFC3501 for anything else you need. I might have to do that sometime. But from what I've seen of IMAP it is more complex than SMTP. POP3 was (though not greatly so). Still, I don't feel I'd want to implement a POP3 tool (and don't need one). An IMAP library might be doable (though not in Perl since I don't know that language and don't have the time to learn it), but the basic just pick up and delete all mail would be sufficient. As suggested by someone else, you can use 'fetchmail' to do that. Normally it delivers using SMTP, but with appropriate flags I believe it can pipe all the retrieved mail to stdout. And if these are all separate mailboxes, POP3 will do for your purposes anyway. Or maybe just hack its source code?
Re: [Dovecot] looking for IMAP testing tool
On Wed, May 19, 2010 at 12:49, Mailing List anonm...@dovecot.org wrote: Hi Phil, I've got an hack piece of regression perl code that we might be a good starting point. I'd love to turn it into a better piece of hack code that we could offer up. It does SSL/TLS. Right now it is pretty basic and forks 1K-10K simultaneous IMAP login/logout connections. Its goal is to find the sink point for the IMAP server and/or test client so that tuning exceeds current usage. Let me know if you are interesting in looking at it and cleaning it up. ---Jack The best I could do with something in Perl is rewrite it into something else. I'm still kinda weak on Python (don't know all the cool tricks, yet), and Pike isn't found universally, so things would need to be in C for me to do it and it to be widely usable. And that's for later when I get some free time.
[Dovecot] looking for IMAP testing tool
I'm looking for an IMAP testing tool, suitable to use with Dovecot IMAP. It needs to support TLS, STARTTLS, and login/authentication. It needs to be able run from command line, shell scripts, and even do so under cron jobs (e.g. a way to supply the password to use w/o a terminal prompt). Typical interactive mail clients just don't cut it (even the text mode ones). One reason is I need to do the tests on a number of machines, under a number of user and domain names, and with a variety of parameters or destinations. This is for a suite of regression tests I am putting together intended to verify that configuration changes do not break things (or unbreak things that are supposed to not work). Anyone ever heard of such a tool? Open source would be preferred (better yet, my favorite programming languages: C, Pike, Python).
Re: [Dovecot] looking for IMAP testing tool
On Tue, May 18, 2010 at 10:17, Steffen Kaiser skdove...@smail.inf.fh-brs.de wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 18 May 2010, Phil Howard wrote: Anyone ever heard of such a tool? Open source would be preferred (better http://search.cpan.org/search?mode=distquery=imap Those all looked like libraries/modules. Any complete commands? Writing Perl code is not an option for me.
Re: [Dovecot] looking for IMAP testing tool
On Tue, May 18, 2010 at 11:42, Hugo Monteiro hugo.monte...@fct.unl.ptwrote: Replying myself in this one. Should have looked a bit further into it. http://imapwiki.org/ImapTest What I'm looking for is not something that just tries commands. What I want to do is actually try to pick up mail and store it in some directory. My script would then check to see if it got the mail it was expecting, or got a failure it was expecting (for a scenario that should fail). Sorry for not being clear on that.
Re: [Dovecot] looking for IMAP testing tool
On Tue, May 18, 2010 at 13:17, Mark Moseley moseleym...@gmail.com wrote: I haven't used it much but it looked useful: http://bc-bd.org/blog/imapfoo/ Hmmm. That (inserting mail into folders) can have it uses. But I'm really looking for something to do, in a very basic way, what is expected of an IMAP client, which is to fetch and read mail. Something that can login and list messages to stdout, or login and fetch a specified message to stdout, would be what I need (and that login might be by clear port 143, STARTTLS on port 143, or plain SSL/TLS on port 993).
Re: [Dovecot] dovecot/deliver ... Can't open log file /var/log/dovecot/error.log: Permission denied
On Fri, May 14, 2010 at 17:48, Noel Butler noel.but...@ausics.net wrote: CDB, oh dear god, you want to go back in time? CDB is no better than any other flatfile based system, it was horrible with qmail and it'll be horrible with anything else above a couple thousand users, you clearly dont add/del users all the time, rebuilding its DB can take some time (I've seen some take 3 minutes, tuff luck if your clients want to add a few users,... so using that is something you cant afford to do as a SP. I see no problem with CDB. I designed my own variation of that a couple decades ago. Mine isn't quite as fast as CDB, but it was along the basic idea. It was very useful in its time. Had CDB not come along, I'd probably have tried to add mine into things like Postfix and Dovecot at some point. MySQL makes it such a dream, even with customers adding aliases and so on, its a simple instruction to mysql via the web portal from them, and using replication means every front end has its own local copy, and able to fallback to the master if for some reason it becomes unavailable (never seen that in all the years been using it tho, but its nice insurance) MySQL (or PostgreSQL, etc) has its place. And for things like CRM with a lot of different aspects, that is the way to go. But even then, I would (and have for other purposes) just export the data out of the SQL database and build a fast index like CDB. However long it takes to build CDB is NOT downtime; it's just lag from data entry to activation. And there are ways to work around that if the lag is an issue, such as having a CDB first, followed by another lookup that may have the most recent data. For example, when the list of new users arrives, add them to a Berkeley DB that is queried next when the CDB has no match. Thus they work even while the CDB (think of it has a static cache) is being rebuilt. One thing I would NOT do is have mail servers hitting the CRM database (or its replicas) directly. It's not a performance issue; it's a security issue. The larger the operation, the more important this is. its your network (I hope for your sake).. its up to you how efficient it is. CDB is very fast.
