Roundcube <-> Dovecot timeouts under very moderate load on big Dovecot nodes
Due to a recent topic regarding the performance of the Roundcube webmailer with Dovecot as a backend, I set up imapproxy successfully, shrinking the response latency of the webmailer heavily, which is great. However, I still encounter strange timeouts on some requests when Roundcube tries to access Dovecot. Be it for authentication or to FETCH data. I run the tests/benchmarks via distributed JMeter from 4 nodes, they access the webmailer which in turn accesses the Dovecot via imapproxy. There are 16 different mailaccounts spread over two dovecot nodes with the first one acting as a proxy if the account resides on the second. The problems start at around 100 concurrent users and gets worse the more users are added. The timeouts make up for a small percentage of the requests (0.0something percent). The timeout is reached after 30 seconds of Dovecot not answering, due to the timeout set in the webserver the webmailer is running on. Now I am wondering how to further debug this and what could be the reason. Some requests towards Dovecot are certainly taking longer than 30 seconds, while the average request takes mostly less than 1. The Dovecot is running on two big hardware machines with 48 cores, 96G memory and SSD storage. The load on the Dovecot nodes while the test is running is minimal. Best regards, M. Schmidt
RE: How do you scale dovecot for good performance with Roundcube webmailer in front? (hitting limits without exhausting resources)
Am 26.01.2023 21:25 schrieb Marc :> > Generally speaking the setup performs OK, I wrote a JMeter Roundcube script to > test the Roundcube. This includes actions like listing mails and fetching > them. I am hitting a somehow dubious limit of 600 req/s though. (This includes > all kinds of HTTP calls to Roundcube, not only the ones triggering IMAP). When > the dovecot becomes unavailable, the performance without mails jumps to 7700 > req/s. Testing is done with 4 JMeter-servers and one client. > Should you not be testing also dovecot performance directly? So you can see what is it's maximum. I can't imagine http interfaces are the bottle neck. Right, I also set up imaptest and tested with that. I seem to hit a limit around 600 auth/s (auth workers 64, hashing algorithm does not seem to matter, tried all 3) and get about 1100 append/s and 1100 fetch/s.But I must say that the imaptest ran in parallel (10) for the append and fetch test on a single machine. Now that I think about it, the auth test may have been limited by the single imaptest instance, as I did not run that in parallel.
How do you scale dovecot for good performance with Roundcube webmailer in front? (hitting limits without exhausting resources)
Good day, I am currently setting up/debugging a webmailer-only setup using Roundcube (latest version) with Dovecot (2.3.20, latest as of now). Generally speaking the setup performs OK, I wrote a JMeter Roundcube script to test the Roundcube. This includes actions like listing mails and fetching them. I am hitting a somehow dubious limit of 600 req/s though. (This includes all kinds of HTTP calls to Roundcube, not only the ones triggering IMAP). When the dovecot becomes unavailable, the performance without mails jumps to 7700 req/s. Testing is done with 4 JMeter-servers and one client. My setup is a Dovecot with MySQL (Percona XtraDB) backend. Mails are encrypted with mail_crypt and EC keys. Authentication is done in parallel (auth_cache_verify_with_workers=yes). Where possible min_available_processes have been set equal to the threads available on the Dovecot machine (64). Hardware is a 64 thread Xeon CPU at 2.10 GHz, 96 GB RAM, SSDs as backing storage IOPS read 4/write 13000. What settings do you recommend and how was your experience with Roundcube and its performance in general (what should be possible with that kind of HW?)? Best regards and thanks in advance!
NFS Config vs Ceph / GlusterFS
That is why I am using mdbox files of 4MB. I hope that should give me hardly any write amplification. I am also seperating between ssd and hdd pools by auto archiving email to the hdd pools I am using rbd. After luminuous I had some issues with the cephfs and do not want to store operational stuff on it yet. I am very interested in that setup, since I am currently planning to reshape my cluster in a similar way (currently from sole distribution via director to distribution + HA). Could you post a short overview (scheme) and some important configurations of your setup? Did you do any performancetesting? Also, when you say rbd in a clustered context, is that one block device per node while the director still spreads the accounts over the nodes? Thanks in advance, M. Schmidt OpenPGP_signature Description: OpenPGP digital signature
Enabling auth_cache_verify_password_with_worker disables proxy mode
Setting "auth_cache_verify_password_with_worker = yes" in order to leverage multiple cores for the Dovecot authentication-process causes Dovecot not to proxy anymore. With debug-logging I figured: auth_cache_verify_password_with_worker = no passdb out: OK 1 user=username host=bla port=10993 ssl=any-cert mail_crypt_global_public_key=key mail_crypt_global_private_key=otherkey hostip=123.123.123.123 proxy pass=pw auth_cache_verify_password_with_worker = yes passdb out: OK 1 user=username The rest seems missing. Dovecot version: v2.3.11.3 Searching for this issue, I found at least two posts mentioning the same symptoms: https://dovecot.org/pipermail/dovecot/2018-April/111583.html https://dovecot.org/pipermail/dovecot/2020-April/118564.html https://listen.jpberlin.de/pipermail/dovecot/2020-April/001915.html (same as the english one from april 2020)
Re: Which allowed services can be defined (imap, pop3, etc.)
On 06/06/2017 03:18 PM, Steffen Kaiser wrote: > (!(service=%s)) > > or better name this attribute > > deniedService Thanks, this is quite helpful already. Regarding the other question about all the services that can be used there, I tried to grep the source code for certain keywords but could not really find anything useful with "service", "services" and some service names (e. g. "imap", "smtp", "pop"). 0xFF379C0C.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
Which allowed services can be defined (imap, pop3, etc.)
Hello, I am using Dovecot with an LDAP-backend for authentication. According to the documentation at https://wiki.dovecot.org/Authentication/RestrictAccess with LDAP and "pass_filter" it is possible to filter allowed services for the user with: pass_filter = (&(objectClass=posixAccount)(uid=%u)(service=%s)) Thats pretty cool. Now, in the LDAP-settings I created corresponding fields (service) and added the allowed services to tthese fields (imap, pop3, etc.). After that change, certain services were not available to that user anymore. So it seems that more services are there and after explicitly setting the services which are allowed, the other, not mentioned services stop to work. Now two questions: Is there any documented, full-featured list of the services that can be added? Is it possible to "flip" that setting so its not allowing certain services but denying the ones that are added to the "service"-fields? Cheers 0xFF379C0C.asc Description: application/pgp-keys signature.asc Description: OpenPGP digital signature
Re: Deleting a mailbox recursively
Hello, is there any update on this topic? Am 11.01.2017 um 08:17 schrieb Malte Schmidt: > Hello and thanks for the answer, > > as of version 2.2.24 this is not working: > > # doveadm mailbox delete -r dove > delete: invalid option -- 'r' > > I crawled through the changelog since 2.2.24 and could not find any > change like this. > Also neither the wiki nor the man-pages mention a "-r"-flag. I am a bit > confused here. signature.asc Description: OpenPGP digital signature
Re: Deleting a mailbox recursively
Hello and thanks for the answer, as of version 2.2.24 this is not working: # doveadm mailbox delete -r dove delete: invalid option -- 'r' I crawled through the changelog since 2.2.24 and could not find any change like this. Also neither the wiki nor the man-pages mention a "-r"-flag. I am a bit confused here. Am 10.01.2017 um 22:55 schrieb Timo Sirainen: > On 9 Jan 2017, at 19.29, Malte Schmidt <i...@vialactea.de> wrote: >> This topic was already discussed on this mailinglist: >> >> https://www.dovecot.org/list/dovecot/2011-November/079491.html >> >> I am also looking for a nice and convenient way of recursively deleting >> a mailbox which uses the mdbox-Format. Since then - was there any progress? > Looks like there's doveadm mailbox delete -r
Deleting a mailbox recursively
This topic was already discussed on this mailinglist: https://www.dovecot.org/list/dovecot/2011-November/079491.html I am also looking for a nice and convenient way of recursively deleting a mailbox which uses the mdbox-Format. Since then - was there any progress? signature.asc Description: OpenPGP digital signature
