Grant, > Anyway. I've done quite a bit of work recently on our front end filter > - a custom developed greylist filter which uses data collected by > amavisd to help decide what to 4XX and what to let through. More > importantly I've just completed updating that filter to make extensive > use of memcached (www.danga.com/memcached/) and wow what a difference > it made to performance. > > I now want to look at whether memcached would work well with bayes and > amavisd. Bayes in particular because I think it is the hardest hitting > database app and amavisd because I'm about to start the upgrade process.
I don't think Bayes is a bottleneck at all. Using MySQL InnoDB to hold a Bayes db, it typically takes 15..35 ms for SpamAssassin to do its bayes lookup, and maybe twice as much for an automatic 'learn' (writing/updating records) when needed. Compared to the rest it is pretty much negligible. Btw, with amavisd-new-2.6.0 and SA 3.3 (cvs) you can see SpamAssassin timing breakdown in the amavisd log, e.g.: Jun 17 15:44:09 patsy amavis[8920]: (08920-21) TIMING-SA total 5478 ms - parse: 1.69 (0.0%), extract_message_metadata: 16 (0.3%), get_uri_detail_list: 2 (0.0%), tests_pri_-1000: 25 (0.4%), tests_pri_-950: 0.67 (0.0%), tests_pri_-900: 0.83 (0.0%), tests_pri_-400: 19 (0.3%), check_bayes: 17 (0.3%), tests_pri_0: 5323 (97.2%), check_spf: 12 (0.2%), poll_dns_idle: 0.81 (0.0%), check_dkim_signature: 1.50 (0.0%), check_razor2: 5022 (91.7%), check_dcc: 192 (3.5%), check_pyzor: 0.02 (0.0%), tests_pri_500: 9 (0.2%), tests_pri_1000: 24 (0.4%), total_awl: 23 (0.4%), check_awl: 10 (0.2%), update_awl: 8 (0.1%), learn: 36 (0.7%), get_report: 1.77 (0.0%) (unlike native amavisd timing report, the timing sections in TIMING-SA may be overlapping, so their sum is typically larger than the actual elapsed time for the entire check - like in the above example where check_razor2 was waiting for its response during the same period that tests_pri_0 was crunching its regexps) > I'm looking for input on this. Has anyone else looked at using > memcached in bayes or amavisd ? Is there perhaps a version already > done ? Does anyone have a view on how effective or perhaps ineffective > it would be ? I'm not aware of any work in this area. The memcached is certainly a nice solution looking for new problems to solve. Instrumenting a berkeleydb bayes backend with memcached might be useful, but given the number of problems with a bdb backend compared to how uneventful life with a sql-based bayes becomes, I doubt it is worth the effort. > My experience with our prefilter leads me to believe it would have > huge impact on bayes performance, and with amavisd in the areas of > lookups for black/white lists, policies etc. My only qualm is really > amavisd because although the cache is miles faster than the database, > the records are small and quite likely cached by the system anyway. Do you have in mind amavisd SQL lookups, or amavisd SQL logging (and pen pals lookups), or SpamAssassin's Bayes and AWL databases? Mark ------------------------------------------------------------------------- Check out the new SourceForge.net Marketplace. It's the best place to buy or sell services for just about anything Open Source. http://sourceforge.net/services/buy/index.php _______________________________________________ AMaViS-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/amavis-user AMaViS-FAQ:http://www.amavis.org/amavis-faq.php3 AMaViS-HowTos:http://www.amavis.org/howto/
