The "heavy" csscleans that I'm running help with my system load, but have caused dspam to be noticeably less good at catching spam. Does anyone here know anything about the performance scaling of the hash driver? Should I take this to the devel list?
Thanks, Peter On Thu, 2011-10-27 at 23:43 -0700, Peter Williams wrote: > On Thu, 2011-10-27 at 17:28 -0700, Peter Williams wrote: > > Hi all, > > > > Around a month ago, I set up a new server and installed dspam on it. > > Things have been running smoothly until today, when suddenly the > > performance of dspam has slowed to a crawl. Even simple messages take a > > few seconds to process (when they used to go by instantaneously), and a > > single HTML or base64-encoded message can send my load average up to 5. > > > > Any thoughts as to how this might have happened? I'm quite sure that the > > character of my email traffic hasn't suddenly changed, and the rest of > > the system seems to be functioning smoothly. Can dspam learn enormous > > tokens, and if it did so, would that slow down subsequent processing? > > Investigating deeper, I think I've crossed some threshold with the hash > driver. I just now tried the (undocumented?) "heavy" option to cssclean, > clearing out some extra tokens from my CSS files, and the performance > has gotten back to where it was before. (I've been running cssclean > daily.) > > (One thing I should have mentioned was that the performance issues are > definitely I/O-related, not CPU-related.) > > I noticed that some of my CSS files have just begun filling up their > second extent, so maybe there's some nonlinearity in the performance > there. The "heavy" cssclean removed the second extent from one user's > file that I checked, but another user's mail is back to good performance > even though the CSS file still has two extents. > > The general tone from web postings is that the hash driver needs a lot > of attention, but most of those postings seem to be a few years old. Is > this sentiment still accurate? > > Peter > > > > > System config info below. > > > > Thanks for any ideas, > > > > Peter > > > > ==================== > > OS: CentOS 6 > > DSpam: 3.10.1 (EPEL build, release 1.el6) > > Context: virtualized server, ~750 MB memory allocated > > > > Incoming mail is accepted by exim, passed to a dspam server on LMTP, > > and delivered by Dovecot communicating over LMTP as well. > > > > Storage driver: libhash_drv > > Other config: > > > > TrainingMode toe > > TestConditionalTraining on > > Feature whitelist > > Algorithm graham burton > > Tokenizer osb > > PValue bcr > > HashRecMax 6291469 > > HashAutoExtend on > > HashMaxExtens 0 > > HashExtentSize 49157 > > HashPctIncrease 10 > > HashMaxSeek 10 > > HashConnectionCache 10 > > Broken case > > > -- Peter Williams / pe...@newton.cx ------------------------------------------------------------------------------ RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 _______________________________________________ Dspam-user mailing list Dspam-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/dspam-user
