Re: Dovecot auth username mapping
Am 2015-07-02 um 01:41 schrieb Laz C. Peterson: I did attempt to switch the PAM/Kerberos authentication to Dovecot LDAP authentication, but now performance is unbelievably slow. Any thoughts to this? In case you have multiple passdb backends, it could be, that LDAP only gets its chance, after PAM did time out. -- peter
Re: Dovecot auth username mapping
Hi Laz,
I’m just wondering… why are you using LDAP and/or PAM to access the MySQL
server? If also the password is stored in the db you could use MySQL directly?
Because then you could use password_query and user_query to actually split the
provided email address into name and domain parts. Then you can lookup each
individually or adjust as needed...
I have something like this:
user_query = SELECT CONCAT('/var/mail/virtual/', SUBSTRING(`mail_addr`,
LOCATE('@', `mail_addr`) +1 ), '/', \
SUBSTRING(`mail_addr`, 1, LOCATE('@', `mail_addr`) -1) ) AS 'home', '1000' AS
'uid', \
'8' AS 'gid', CONCAT('*:bytes=', `quota`, 'M') AS 'quota_rule' FROM
`mail_users` \
WHERE `mail_addr` = '%u' AND `status` = 'ok' AND `mail_type` LIKE '%%_mail%%‘
With an SQL statement you could even use sub-selects and whatnot to do
complicated things. Perhaps you could do something similar with the LDAP string
but I never used LDAP that much…
Philon
Am 02.07.2015 um 02:27 schrieb Laz C. Peterson l...@paravis.net:
It’s actually unbelievable how much slower LDAP auth is than PAM. Does
anyone have any suggestions how I can improve Dovecot LDAP auth? I have
tried caching authentications and that doesn’t help either.
~ Laz Peterson
Paravis, LLC
Ph: 951.319.3240 x201
On Jul 1, 2015, at 4:41 PM, Laz C. Peterson l...@paravis.net wrote:
Thank you for the response Axel. I will look into that.
I did attempt to switch the PAM/Kerberos authentication to Dovecot LDAP
authentication, but now performance is unbelievably slow. For example, with
PAM/Kerberos, a user can log into webmail and have all of their
emails/folders showing almost immediately. When using Dovecot LDAP, it
takes literally 8-10 seconds to see the same thing.
I was hoping that was a possible replacement for this, but my goodness it
was so incredibly slow! This would definitely be an option though, as it
does serve the purpose. I just can’t figure out how to fix the performance
issue. Any thoughts to this?
~ Laz Peterson
Paravis, LLC
Ph: 951.319.3240 x201
On Jul 1, 2015, at 3:24 PM, Axel Luttgens axel.luttg...@skynet.be wrote:
Le 1 juil. 2015 à 04:38, Laz C. Peterson
a écrit :
I have an interesting case here …
Virtual mailboxes, domain/username/aliases stored in MySQL, authentication
done using PAM. PAM authenticates through Kerberos, which are internal
realms and not the email domains — for example, my username would be
laz@PARAVIS.LOCAL mailto:laz@PARAVIS.LOCAL and my email address would be
l...@paravis.net mailto:l...@paravis.net.
All of this works just fine. But what I want to do is allow the users to
log in using their email address and not their full Kerberos name. It is
becoming laborious to help the users understand the difference between
their username@LOCAL.REALM and username@email.address
mailto:username@email.address and why we have to have two separate
identities that mean the same thing.
I have the SQL statements to convert either the Kerberos login or the
email address to the actual Kerberos login (so they may use either). But
I cannot seem to figure out how to get Dovecot to acknowledge this as the
mapped username.
I’m sure there has to be a way. Any help will be greatly appreciated.
Thank you!
Hello Laz,
I fear you’ll have to resort to CheckPassword
(http://wiki2.dovecot.org/AuthDatabase/CheckPassword) or something similar.
Indeed, your MySql database may contain everything needed to convert email
addresses to kerb login (and vice-versa), but Dovecot’s PAM interface
understandably just knows about a (login, password) pair, where the login
is the one provided by the user wanting to log in.
That said, I hope to be wrong,
Axel
Re: duplicate namespace prefix
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Thu, 2 Jul 2015, Daniel Tröder wrote:
Am 02.07.2015 um 00:51 schrieb jjhoffart:
Hello,
I am in the process of setting up a server that is running to
invocations of dovecot. One of the invocations is acting as a backend and the
other as a Director. I believe I have most of the configuration complete but I
keep running into the following error:
Jul 01 14:17:04 lda(postmas...@mydomain.com: Error: user postmas...@mydomain.com:
Initialization failed: namespace configuration error: Duplicate namespace prefix:
Jul 01 14:17:04 lda(postmaster@åmydomain.com: Fatal: Invalid user settings.
Refer to server log for more information.
Not sure where to go from here and most of my searches on the error have lead
me to dead ends hoping someone can help me out.
Thanks.
Each namespace must have a different prefix. The only namespace with an
empty prefix is the private one for the users inboxes. Configure a
different prefix one for each namespace. http://wiki2.dovecot.org/Namespaces
namespace foo {
and don't forget to name the other namespaces differently ;-)
separator = /
prefix = foo/ --- prefix must end in separator symbol
list = children
location = maildir:/var/spool/...
}
- --
Steffen Kaiser
-BEGIN PGP SIGNATURE-
Version: GnuPG v1
iQEVAwUBVZTcmHz1H7kL/d9rAQJHAAf/ZscGIsJD71PvDJGuMY/gPvnSb740UMlp
OQXFc3Wi1lCmv1s4G25N2S3eQDQE1wGvh4Rd55KAiZrhnSWKcx4oDTeT6zuO0WI3
EX2cg4GuPza4aZaCa+tCngRz6NTlVtES64w8SPiC5uq7kR2Mya9mz+XFtY8DZ8Zc
eUeANf3OhiBEmLNOvTQD7qfCMoByCmUNghpw3MZLT5hQyxhfiGcmTGPL0L81Jz6e
o9hSLhkKOg2v55QgvarTLwZk3Xaiop5QM/K0+Nx3+5G2ROZoUOaGodI5SneVyeR1
AXiqV1RT/XliK71La6yRgh3e5F2mKEfEL8fvZ+exspVdG5FzGgL96w==
=p6w5
-END PGP SIGNATURE-
Re: duplicate namespace prefix
Am 02.07.2015 um 00:51 schrieb jjhoffart:
Hello,
I am in the process of setting up a server that is running to
invocations of dovecot. One of the invocations is acting as a backend and the
other as a Director. I believe I have most of the configuration complete but
I keep running into the following error:
Jul 01 14:17:04 lda(postmas...@mydomain.com: Error: user
postmas...@mydomain.com: Initialization failed: namespace configuration
error: Duplicate namespace prefix:
Jul 01 14:17:04 lda(postmaster@åmydomain.com: Fatal: Invalid user settings.
Refer to server log for more information.
Not sure where to go from here and most of my searches on the error have lead
me to dead ends hoping someone can help me out.
Thanks.
Each namespace must have a different prefix. The only namespace with an
empty prefix is the private one for the users inboxes. Configure a
different prefix one for each namespace. http://wiki2.dovecot.org/Namespaces
namespace foo {
separator = /
prefix = foo/ --- prefix must end in separator symbol
list = children
location = maildir:/var/spool/...
}
Daniel
signature.asc
Description: OpenPGP digital signature
Re: Using PAM and passwdfile together
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
On Wed, 1 Jul 2015, Jim Garrison wrote:
I have some local users and some Postfix virtual mailboxes. The config
currently has:
# 2.0.9: /etc/dovecot/dovecot.conf
# OS: Linux 2.6.32-504.12.2.el6.x86_64 x86_64 CentOS release 6.6 (Final)
passdb {
driver = pam
}
passdb {
args = scheme=MD5 username_format=%u /etc/dovecot/auth/%d.passwd
driver = passwd-file
}
Each time a virtual mailbox user logs in, PAM writes a set of
Authentication Failure messages to /var/log/secure when it attempts
to find the virtual user, which it then successfully authenticates
in the passwd-file. Is there a way to prevent PAM from loggin this
spurious error and having dovecot log an authentication failure only if
BOTH methods fail?
Do you have users via PAM and in the passwd-file?
If not or if it does not harm, reverse the order of both passdb's .
- --
Steffen Kaiser
-BEGIN PGP SIGNATURE-
Version: GnuPG v1
iQEVAwUBVZTaGnz1H7kL/d9rAQLKOAgAoNMgpf5r3E/9QQ3CIqa6gtFQyr3ivWRO
j+2JMT63xwJyFNuot3ZRabUr5qNQvQPm+52cZPkX7cBqH+0fB2iiHwIstqaPWHfc
M4iVnfohAonO/8thefmjCTbmx086RD3X7EYmTBNrNxw4gp57Lppz6mgsZzKBoeZ8
1H1FDTWxbikQ8ufb8woQgBcXm+LmT0VTpGpMVEndazFhwVqsxV4jHvrVi+OhbTup
0zllqciQZmat06U/hX3F5oK9L6ZXaCpbEVUcj5Zx7QA8CP7gK6ONNmme5noTLAO8
KMTYgYSNACFRWCAjye4I+djkCDJ5EWvf8M/b+czmumzRlRXmuBanFQ==
=zuRO
-END PGP SIGNATURE-
Re: the file dovecot-uidlist.lock appear then deadlock
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 2 Jul 2015, 刘莹莹 wrote: sometimes i can't use imap command to get mails, when this problem appear i find out the file dovecot-uidlist.lock appear at the same time . I don't know Why is this happening ? can you help me ? + post the output of: doveconf -n + post the last entries in the log and + explain what can't use imap command means, do you login with a mail client or do you use the command line + when this happens, please run post: doveadm who - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEVAwUBVZUGGHz1H7kL/d9rAQKeZQf9Gk2yYVgAwsyXTxOww9P6kUA7PgUJpgTQ RQGQtrBokxGXcUzBTVqLoJs1rpI+qsF0AxR2yTKwhTnqbzGnbToxlRddyQVkA+wE 6WGVbrQcX58mXYjRFFyig5HPUPHgmVBDUZudVgVCQIYnDe/zs3kIE3Nh5JLhblMv vNGJeHpzpG1yg/c7gOjhFsqNbaFeXecVTEpRlO+PwnkdogYqfUug0ul9MfHzQRi1 /KM9DO7SkPyYDG6KZGsxKcAOMoGRAmSw9I2taYtsBF3yFFV058y/+vRFHxdHTLkb taust3ggt6UhSPOsmkdO8WD5JwYG7VmZ9XBJaWTaqjHInzcXyUhwIQ== =/hUd -END PGP SIGNATURE-
the file dovecot-uidlist.lock appear then deadlock
hi: sometimes i can't use imap command to get mails, when this problem appear i find out the file dovecot-uidlist.lock appear at the same time . I don't know Why is this happening ? can you help me ? thanks !
RE: Outlook 2013 not fetching new mail/synchronization issues
Unfortunately disabling antivirus did not help. I also set IMAP on 143 TLS, removed Root folder path and left everything unchecked in outlook delete. Emails are still not fetching. Any other ideas ? Pacher Dragos -Original Message- From: dovecot [mailto:dovecot-boun...@dovecot.org] On Behalf Of Robert Schetterer Sent: 30 June 2015 18:05 To: dovecot@dovecot.org Subject: Re: Outlook 2013 not fetching new mail/synchronization issues Am 30.06.2015 um 13:06 schrieb Dragos Pacher: There is also an antivirus running on client with antispam module. I will try to disable it and see if there is any change. Pacher Dragos yeah most problems are depend to such software Best Regards MfG Robert Schetterer -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: Outlook 2013 not fetching new mail/synchronization issues
On Thu, 2 Jul 2015 13:16:06 +0300, Dragos Pacher stated:
I also set IMAP on 143 TLS, removed Root folder path and left everything
unchecked in outlook delete.
Emails are still not fetching.
Any other ideas ?
Have you use any other MUA to download emails prior to Outlook? Is any other
MUA running in the background when attempting to use Outlook? I have no idea
at this point what your problem is. I have a basic Dovecot configuration, and
it seems to work fine with Outlook 2013. This is the Dovecot config:
# 2.2.18: /usr/local/etc/dovecot/dovecot.conf
# Pigeonhole version 0.4.8 (0c4ae064f307+)
# OS: FreeBSD 10.1-RELEASE-p10 amd64 ufs
auth_mechanisms = plain login cram-md5 digest-md5
lda_mailbox_autocreate = yes
lda_mailbox_autosubscribe = yes
log_path = /var/log/dovecot.log
mail_location = maildir:/var/mail/vmail/%d/gerard
managesieve_notify_capability = mailto
managesieve_sieve_capability = fileinto reject envelope encoded-character
vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy
include variables body enotify environment mailbox date ihave duplicate
namespace inbox {
inbox = yes
location =
mailbox Drafts {
special_use = \Drafts
}
mailbox Junk {
special_use = \Junk
}
mailbox Sent {
special_use = \Sent
}
mailbox Sent Messages {
special_use = \Sent
}
mailbox Trash {
special_use = \Trash
}
prefix =
}
passdb {
args = username_format=%u /usr/local/etc/dovecot/user/passwd
driver = passwd-file
}
passdb {
args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/user/passwd
driver = passwd-file
}
passdb {
args = scheme=CRYPT username_format=%u /usr/local/etc/dovecot/user/passwd
driver = passwd-file
}
plugin {
sieve_global_dir = /usr/local/etc/dovecot/sieve/
sieve_global_path = /usr/local/etc/dovecot/sieve/default.sieve
}
postmaster_address = postmas...@seibercom.net
service auth {
unix_listener /var/spool/postfix/private/auth {
group = postfix
mode = 0666
user = postfix
}
unix_listener auth-userdb {
group = vmail
mode = 0666
user = vmail
}
}
service imap-login {
inet_listener imap {
port = 143
}
inet_listener imaps {
port = 993
ssl = yes
}
}
service pop3-login {
inet_listener pop3 {
port = 110
}
inet_listener pop3s {
port = 995
ssl = yes
}
}
ssl_cert = /etc/ssl/certs/dovecot.pem
ssl_cipher_list = AES128+EECHD:AES128+EDH
ssl_key = /etc/ssl/private/dovecot.pem
ssl_prefer_server_ciphers = yes
ssl_protocols = !SSLv2 !SSLv3
userdb {
args = username_format=%u /usr/local/etc/dovecot/user/passwd
default_fields = uid=vmail gid=vmail
driver = passwd-file
}
userdb {
driver = passwd
}
userdb {
args = username_format=%u /usr/local/etc/dovecot/user/passwd
driver = passwd-file
}
version_ignore = yes
protocol lda {
mail_plugins = sieve
}
Have you tried using tripwire to see what is happening or enlised the help
of anyone on those MS Forums I sent you? There is a tech forum for Outlook.
I don't have the URL handy right now, but I will look it up.
BTW, did you make sure Outlook has the latest updates installed? The latest
version is 15.0.4727.1003
I believe you did set Enable troubleshooting under File Options
Advanced (near the bottom of the list).It requires a restart of Outlook.
Have you right clicked on the folder name and then IMAP Folders and
seeing if al of the folders are present? You could try the same thing only
check Update Folder List. Also check Account Property's Rules
Alerts. A faulty rule might be causing the problem. Outlook will usually list
a faulty or broken rule in red.
--
Jerry
pgpLDm7G1j75n.pgp
Description: OpenPGP digital signature
Re: Dovecot auth username mapping
Peter, Yes that is a possibility. I will try disabling PAM (or switching the auth order) and see if that makes a difference. Thanks for the suggestion! ~ Laz Peterson Paravis, LLC Ph: 951.319.3240 x201 On Jul 1, 2015, at 11:34 PM, Peter Chiochetti p...@myzel.net wrote: Am 2015-07-02 um 01:41 schrieb Laz C. Peterson: I did attempt to switch the PAM/Kerberos authentication to Dovecot LDAP authentication, but now performance is unbelievably slow. Any thoughts to this? In case you have multiple passdb backends, it could be, that LDAP only gets its chance, after PAM did time out. -- peter
autocreating folders with special chars stored in mysql-userdb
Hello, i use dovecot 2.2.18 and do some tests with an utf8 mysqlbackend. I use MySQL to authenticate Users and for retrieving some individual settings like autocreating folders etc. by userdb-settings in user-query. Yet no problems with standard language. Now i try to provide userdb-settings in exotic languages. My MySQL table uses utf8 charset. I store some data in my userdb-column. Here an polish example: namespace/inbox/mailbox=example namespace/inbox/mailbox/example/name=postępować namespace/inbox/mailbox/example/auto=subscribe Result is a new folder in dovecot, but in client every special char ę and ć is replaced with a ? I seems that at some point a conversion to latin1 is performed which ruins the utf8 chars. Creating a folder postępować by imap-client is no problem. Hmm, what should i do now? I dont know how to influence dovecot to process retrieved utf8 data furthermore as utf8. Which charset is using dovecot when connecting to mysqldb and how to take influence? Thanks, Hajo
Re: imapc fetch optimization
On 05/04/15 17:45, Timo Sirainen wrote: On 04 May 2015, at 16:59, Nagy, Attila b...@fsn.hu wrote: Hi, On 05/04/15 10:59, Timo Sirainen wrote: On 28 Apr 2015, at 23:49, Nagy, Attila b...@fsn.hu wrote: Hi, imapc does a lot of UID FETCH $UID (BODY.PEEK[]), which is nice, because it works even with the dumbest IMAP server, altough it really kills performance, especially on high latency lines. I wonder: if IMAP servers can effectively handle boundless fetches (like a list with all wanted UIDs, or simply 1:* if all are needed), do you see this as a good addition to develop? Set mail_prefetch_count = 10 or 100 or something and it'll do larger FETCHes. The higher the value, the more memory/disk space is used for storing the received mails. I'm aware of that, but it doesn't, or at least not always. For example Thunderbird issues this: 13 UID fetch 333574:333601,333630:333801 (UID RFC822.SIZE FLAGS BODY.PEEK[HEADER.FIELDS (From To Cc Bcc Subject Date Message-ID Priority X-Priority References Newsgroups In-Reply-To Content-Type Reply-To)]) Dovecot does this to the IMAP backend 18 UID FETCH 333574 (BODY.PEEK[]) Oh, there were several bugs related to that. Fixed: http://hg.dovecot.org/dovecot-2.2/rev/8f20aa806bcc http://hg.dovecot.org/dovecot-2.2/rev/d350a23207c2 http://hg.dovecot.org/dovecot-2.2/rev/8c49fb6d789b I've just tried 2.2.18, which has the above modifications, but the effect is the same. I have these in the config: # doveconf -n | egrep 'prefe|imapc' imapc_features = rfc822.size fetch-headers imapc_host = 10.3.34.12 mail_location = imapc:~/imapc mail_prefetch_count = 100 With an empty imapc directory a POP3 login yields these on the backend IMAP server: 2 LOGIN user pass 1 LIST 3 SELECT INBOX 4 NOOP 5 UID FETCH 1:* (FLAGS) 6 UID FETCH 2 (RFC822.SIZE) 7 UID FETCH 3 (RFC822.SIZE) 8 UID FETCH 4 (RFC822.SIZE) 9 UID FETCH 5 (RFC822.SIZE) 10 UID FETCH 6 (RFC822.SIZE) 11 UID FETCH 7 (RFC822.SIZE) 12 UID FETCH 8 (RFC822.SIZE) [...] so on, for every message... The response times for each of the FETCHes are around 13-16 ms, so logging into dovecot for the first time takes more than a minute with 5000 mails in the INBOX and a half day with around a million... It would be good to have a: 6 UID FETCH 1:* (RFC822.SIZE) or even a: 5 UID FETCH 1:* (FLAGS RFC822.SIZE) if needed (dovecot first fills up its cache, or other cases when the full list is needed) If dovecot would issue the latter for this 5000-mail inbox, the first fetch (and POP login) would take only 500 ms instead of the 75 seconds that it takes now... Could you please tweak this some more? :) Thanks,
Re: Dovecot auth username mapping
Ahh Peter, good call on this one! beating head into deskpausebeating head into desk againthumbs up So after playing around with the order of authentication in Dovecot, you are correct, the PAM timeout was causing the holdup. I guess since PAM has no way of looking up whether or not a user exists prior to authenticating, this is causing the hiccup, versus LDAP which can search for a user’s existence prior to the auth. Switching these around, I notice almost *no* degradation in performance for PAM authentications, and the LDAP authentications run smooth as I would hope them to. Awesome, so now we have our solution! (I think.) Gotta say, a lot of love goes out to the Dovecot community (especially Timo!) for all the inspiration and help that I’ve received. Dovecot is a great app and this community is the backbone of it all. Cheers to all! Thanks again. ~ Laz Peterson Paravis, LLC Ph: 951.319.3240 x201 On Jul 2, 2015, at 6:25 AM, Laz C. Peterson l...@paravis.net wrote: Peter, Yes that is a possibility. I will try disabling PAM (or switching the auth order) and see if that makes a difference. Thanks for the suggestion! ~ Laz Peterson Paravis, LLC Ph: 951.319.3240 x201 On Jul 1, 2015, at 11:34 PM, Peter Chiochetti p...@myzel.net wrote: Am 2015-07-02 um 01:41 schrieb Laz C. Peterson: I did attempt to switch the PAM/Kerberos authentication to Dovecot LDAP authentication, but now performance is unbelievably slow. Any thoughts to this? In case you have multiple passdb backends, it could be, that LDAP only gets its chance, after PAM did time out. -- peter
Re:Re: the file dovecot-uidlist.lock appear then deadlock
dovecot -n log_path: /var/log/maillog protocols: pop3 pop3s imap imaps bport ssl_cert_file: /etc/pki/NSMail/SSL.cert ssl_key_file: /etc/pki/NSMail/SSL.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(pop3): /usr/libexec/dovecot/pop3-login first_valid_uid: 150 mail_uid: 199 mail_gid: 199 mail_location: maildir:/%Lh/%Ld/%Ln/:INDEX=/%Lh/%Ld/%Ln/ mmap_disable: yes mail_nfs_index: yes mail_drop_priv_before_exec: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(pop3): /usr/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota zlib mdec imap_acl acl autocreate expire mail_plugins(imap): quota imap_quota zlib mdec imap_acl acl autocreate expire mail_plugins(pop3): quota zlib mdec expire mail_plugin_dir(default): /usr/lib64/dovecot/imap mail_plugin_dir(imap): /usr/lib64/dovecot/imap mail_plugin_dir(pop3): /usr/lib64/dovecot/pop3 pop3_no_flag_updates(default): no pop3_no_flag_updates(imap): no pop3_no_flag_updates(pop3): yes pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh namespace:.. The exceptions log Jun 29 09:44:49 IMAP(pub...@test.com): Error: *** %n in writable segment detected *** Jun 29 09:44:49 dovecot: Error: child 138430 (imap) killed with signal 6 (core dumped) Jun 29 09:44:50 mail postfix/smtpd[58328]: warning: hostname[127.0.0.1]: SASL LOGIN authentication failed: UGFzc3dvcmQ6 Jun 29 09:44:50 mail postfix/smtpd[58328]: lost connection after AUTH from hostname[127.0.0.1] Jun 29 09:44:50 mail postfix/smtpd[58328]: disconnect from hostname[127.0.0.1] Jun 29 09:44:51 mail postfix/smtpd[29019]: connect from hostname[127.0.0.1] when there are too many new messages in INBOX (about 4000 new messages) , I can't open INBOX by webmail to check new e-mails . at the same time there is a lock file named dovecot-uidlist.lock still in maildir directory . I have never see this file ,I guess dovecot-uidlist is locked by dovecot-uidlist.lock and no longre unlocked so dovecot-uidlist can't be operation then I unable to access mailbox. this is my guess , dovecot-uidlist.lock appears under what conditions? At 2015-07-02 17:36:23, Steffen Kaiser skdove...@smail.inf.fh-brs.de wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Thu, 2 Jul 2015, 刘莹莹 wrote: sometimes i can't use imap command to get mails, when this problem appear i find out the file dovecot-uidlist.lock appear at the same time . I don't know Why is this happening ? can you help me ? + post the output of: doveconf -n + post the last entries in the log and + explain what can't use imap command means, do you login with a mail client or do you use the command line + when this happens, please run post: doveadm who - -- Steffen Kaiser -BEGIN PGP SIGNATURE- Version: GnuPG v1 iQEVAwUBVZUGGHz1H7kL/d9rAQKeZQf9Gk2yYVgAwsyXTxOww9P6kUA7PgUJpgTQ RQGQtrBokxGXcUzBTVqLoJs1rpI+qsF0AxR2yTKwhTnqbzGnbToxlRddyQVkA+wE 6WGVbrQcX58mXYjRFFyig5HPUPHgmVBDUZudVgVCQIYnDe/zs3kIE3Nh5JLhblMv vNGJeHpzpG1yg/c7gOjhFsqNbaFeXecVTEpRlO+PwnkdogYqfUug0ul9MfHzQRi1 /KM9DO7SkPyYDG6KZGsxKcAOMoGRAmSw9I2taYtsBF3yFFV058y/+vRFHxdHTLkb taust3ggt6UhSPOsmkdO8WD5JwYG7VmZ9XBJaWTaqjHInzcXyUhwIQ== =/hUd -END PGP SIGNATURE-
Scalability with high density servers and proxies, TCP port limits
Hello, first post in 3 years, kinda shows how painless Dovecot is. ^o^ Also this isn't really a dovecot issue, alas it's involved and since there are some large scale implementations of it I hope somebody here has some insights I might have missed. Currently we're running this setup: 1. LVS (DR mode) in a HA configuration (2 node cluster) 2. Dovecot in proxy mode on a 2 node cluster 3. Dovecot on actual mailbox servers (dual node DRBD clusters) There are about 500k users, but most of them use POP3, so there are usually less than 6k IMAP sesions at any given time. This is about to change, I'm looking at potentially millions of users who will have all semi-permanent IMAP sessions. We already have a pure SSD based mailbox cluster and based on the experiences with that another one is on order that will be able to easily handle about 500k users with regards to IOPS and other needs. However there's the issue of having all these concurrent IMAP sessions. Namely, running out of ephemeral ports. Lets assume 2 million users and 50k ports per IP and revisit the setup above. 1. LVS should have no problem, from experience and tests I expect a well tuned and spec'ed machine to handle millions of connections. This is in DR mode, in NAT mode I assume things would run into a wall a lot quicker. But even if LVS should run out of steam, there's a wide selection of high capacity load balancers available. 2. Here is where the fun starts. Each IMAP session that gets proxied to the real mailbox server needs a port for the outgoing connection. So to support 2 million sessions we need 40 IP addresses here. Ouch. And from a brief test having multiple IP addresses per server won't help either (Dovecot unsurprisingly picks the main IP when establishing a proxy session to the real mailbox), at least not with just one default GW. 3. All of this gets repeated on the actual mailbox servers, by either having a lot of low density servers or (preferably) high density servers with multiple IP addresses. Am I on track so far or missing something obvious? How many concurrent connections do you (hello Timo) think dovecot in proxy mode can handle? High performance mode of course in this case. I'm interested in internal limitations, assume that CPU and RAM are amply supplied. Any and all feedback is appreciated. Regards, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/
Re: Scalability with high density servers and proxies, TCP port limits
On Fri, 03 Jul 2015 07:05:43 +0200 Urban Loesch wrote: Hi, Am 03.07.2015 um 05:14 schrieb Christian Balzer: 2. Here is where the fun starts. Each IMAP session that gets proxied to the real mailbox server needs a port for the outgoing connection. So to support 2 million sessions we need 40 IP addresses here. Ouch. And from a brief test having multiple IP addresses per server won't help either (Dovecot unsurprisingly picks the main IP when establishing a proxy session to the real mailbox), at least not with just one default GW. If I remeber correctly there is a config option in dovecot 2.x where you can set the ip addresses which dovecot should use for outgoing proxy connections. Sorry, but I can't remeber the option. Looking at the documentation on the Wiki I was going to say That won't help, as it says address. http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/Proxy But since that page is rather terse, I looked up the changelog and found that it indeed was added for use cases like mine: http://www.dovecot.org/list/dovecot-cvs/2014-June/024574.html Unfortunately the latest dovecot version in Debian is 2.2.13... Additionally this still leaves the actual mailbox servers, which in my case will need to be able to handle more than 50k sessions as well. Thanks for the info, Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Global OnLine Japan/Fusion Communications http://www.gol.com/
Re: Scalability with high density servers and proxies, TCP port limits
Hi, Am 03.07.2015 um 05:14 schrieb Christian Balzer: 2. Here is where the fun starts. Each IMAP session that gets proxied to the real mailbox server needs a port for the outgoing connection. So to support 2 million sessions we need 40 IP addresses here. Ouch. And from a brief test having multiple IP addresses per server won't help either (Dovecot unsurprisingly picks the main IP when establishing a proxy session to the real mailbox), at least not with just one default GW. If I remeber correctly there is a config option in dovecot 2.x where you can set the ip addresses which dovecot should use for outgoing proxy connections. Sorry, but I can't remeber the option. Best Urban
