Re: BayesStore in Redis DB

[Jan Hejl]

Dne 27.3.2012 15:52, Benny Pedersen napsal(a):

Den 2012-03-27 13:29, Jan Hejl skrev:

There are few reasons for implementing Redis DB for me (my company)
may be not all of it is objective because i like these key value
storages.

okay

1) Simplicity - Redis DB is simple to use and to maintain and is much
more complex then other key value storages - remember that first
storages were key value (BDB, etc.).

berkdb is on its way out on gentoo, that include the mysql support for it aswell, redis db is not currently in gentoo portage so i cant test it atm

redis is in gentoo portage. i wrote this plugin on gentoo system :-) gentoo portage also contains redis-py client for python

2) Scaleability - Redis DB is pretty highly scalable, and the purpose
of this plugin is tu use Bayes for high performance nodes. In future
there should be lots of new things about Redis functionality, so I
thought, that i would be nice to give it a try.

yep, hope your work will be part of spamassassin if it turns out good, mysqltuner is helpfull for me, since my server only have 1.2G ram, yes ram is cheap, but not on old servers

3) Resources - i did some short tests about memory consuption and i
get to the 10% or less of memory consuption with comparsion to MySQL
engines. This reason may be odd, i have to make much more tests.

super i would like to change here alone for this reason

For next two months we planned few test scenarios, so i'll let you know then.

4) Autoexpire - SEEN table is not expiring and this table grows fast.
If you learn your bayes with 50000 emails per day, it grows into
pretty big monster after few years, and there's no mechanism keeping
time signature of seen entries. With Redis DB you can set EXPIRE time
of SEEN key (it's also implemented in this plugin) and you don't have
to care about anything else.

yes the seen table can be modified to support expire or simply cronned to be deleted, i do it as here with only holds 24 hours last records

sure i agree, but you have to execute cron task which can leads into crash on system with big DB data and higher load caused by higher traffic. For Redis it is more native.

These are my reasons, but I understand if you rather use simplier way
with MySQL memory engine. The point is that the plugin is almost done
so why not make it better?

but according to my reading you do more in the perl code ? :)

sure, if ram speed was demended one could make startup init for mysql to alter table engine memory, and on shutdown alter table engine myisam

Sorry I don't understand this question. More inside perl module code?

smime.p7s
Description: Elektronický podpis S/MIME