Well, I think now I didn't understood your first message.... 8-( What I can say is that there are a number of servers running this without problem since 2004. The biggest one I have some feedback handles almost 30.000 messages an hour.
Surely, this thread model can handle much more traffic than original threads model, but it to has a limit somewhere. The problem there is that this new limit shall be evaluated by someone : libmilter or the filter. The solution I proposed is OS and filter independent. So there are things which can be done at libmilter level and others by the filter. A filter shall eventually be able to evaluate resources consumption and behave adequately. So, if the filter says - sorry, I'm not able to handle any other new connection now, this is better than accepting and crashing. This is usually what's done in real life with real people. Limit the number of threads isn't a good idea, unless the filter (clamav-milter or other) can limit the amount of time allocated to each task (e.g. message content scanning). If you don't do that, you create a DoS vulnerability. If there are new ideas to improve this feature, please let me know. Regards, OBS : If you define _FFR_WORKER_MODEL, SM_CONF_POLL is also defined (libmilter.h) : #if _FFR_WORKERS_POOL /* SM_CONF_POLL shall be defined with _FFR_WORKERS_POOL */ # if !SM_CONF_POLL # define SM_CONF_POLL 1 # endif /* SM_CONF_POLL */ #endif /* _FFR_WORKERS_POOL */ Jose-Marcio Martins da Cruz wrote: > That's already done inside libmilter : > > With sendmail 8.14.0, you shall compile libmilter with : > > APPENDDEF(`conf_libmilter_ENVDEF',`-D_FFR_WORKER_MODEL') > > This will switch the libmilter thread model to a pool of workers instead > of one thread per sendmail connection. And the number of threads in the > filter will be the number of threads really doing something... > > This makes a very big difference on systems handling a big number of > simultaneous connections. > > Here are some old slides (2003) from an old presentation I've done at > sendmail about that : > > http://j-chkmail.ensmp.fr/papers/workermilter.pdf > > Well... I've got an award from sendmail... 8-) > > http://www.j-chkmail.org/wiki/doku.php/award > > > Ernest Wypierowski wrote: >> Good evening, >> >> version: >> libmilter - sendmail-8.14.2 >> clamav-milter - clamav-0.91.2 >> >> >> If you configure libmilter library with using poll() rather than >> select() and use workers pool you might hit problem related with locking >> up your clamav-milter. >> >> To reproduce this kind of behavior you should ran clamav-milter in 32 >> bit mode, lower max-children and set timeout to some rather high value. >> When number of simultaneous connections exceed max children then every >> new connection will equal one thread. Going step further, when threads >> allocate all virtual memory available for clamav-milter workers from >> libmilter will not have space to execute (clamav-milter: LAUNCH_WORKER >> error: Cannot allocate memory). >> >> Daemon clamav-milter will not answer with temporary error but will >> keep going with blocking threads/connections. >> >> How to solve this kind of problem? Does not clamav-milter is prepare for >> workers from libmilter? >> >> Most interesing is how to limit clamav-milter's resource based on thre- >> ads. In described situation number of threads are not equal number of >> connections, limiting it with dont-wait threads is not a good idea. >> Maybe there is a way to limit numbers of threads that workers can >> consume? >> >> Best regards, >> Ernest Wypierowski >> _______________________________________________ >> http://lurker.clamav.net/list/clamav-devel.html >> Please submit your patches to our Bugzilla: http://bugs.clamav.net >> > > -- --------------------------------------------------------------- Jose Marcio MARTINS DA CRUZ http://j-chkmail.ensmp.fr Ecole des Mines de Paris 60, bd Saint Michel 75272 - PARIS CEDEX 06 mailto:[EMAIL PROTECTED] _______________________________________________ http://lurker.clamav.net/list/clamav-devel.html Please submit your patches to our Bugzilla: http://bugs.clamav.net
