On Fri, Dec 20, 2013 at 07:35:40AM -0500, Ludovic Marcotte wrote:
> There are a couple of things you should look at for under-performing
> sogod child processes:
>
> 1. make sure your IMAP server is fast. Look at Cyrus IMAP Server or
> Dovecot with indexes. You might also want to use imapproxy in
> between SOGo and your imap server
> 2. make sure your LDAP server is properly indexed - see the SOGo
> installation guide for details
> 3. make sure the number of sogod workers X the value of SxVMemLimit is
> lower than the amount of RAM you have - so you don't start swapping
> 4. tune your database server - especially if you do use MySQL
Recently, we discovered another aspect that needs to be tuned
on the web side.
We had a performance problem crippling SOGo for several days.
The parent process was waiting a long time for unresponsive children.
Lots of kernel messages filled the log saying:
TCP: Possible SYN flooding on port 20000. Sending cookies.
Strace showed a strange behaviour: a child process accepts an Nginx
connection, and then blocks polling() on the socket for over a minute.
Capture trafic on the loopback interface, we saw TCP retransmissions
between Nginx and SOGo.
With the help of the following page
http://blog.dubbelboer.com/2012/04/09/syn-cookies.html
we understood the problem and found a solution.
Nginx was hammering SOGo too fast and saturating the listen queue.
It turned out that the listen backlock was too low;
Unless specified, WOListenQueueSize defaults to 5 :
(in _prepareListeningSocket,
sope/sope-appserver/NGObjWeb/WOWatchDogApplicationMain.m
backlog = [[NSUserDefaults standardUserDefaults]
integerForKey: @"WOListenQueueSize"];
if (!backlog)
backlog = 5;
[listeningSocket listenWithBacklog: backlog];
(in sope/sope-appserver/NGObjWeb/Defaults.plist)
WOListenQueueSize = 5;
This value is insufficient in a large SOGo installation :
we had enough workers to handle the load.
We set WOListenQueueSize to 511 in ~sogo/GNUstep/Defaults/.GNUstepDefaults
We also modified the following sysctl parameters :
net.core.somaxconn = 511
net.ipv4.tcp_max_syn_backlog = 4096
I am not sure about how much we should really allow
(32 ? 128 ?). But this fixed our problem.
--
Jean BENOIT
--
[email protected]
https://inverse.ca/sogo/lists
