Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Henrik K]

On Tue, Feb 09, 2016 at 08:27:08AM +0100, Benning, Markus wrote:
> On 2016-02-05 16:25, Henrik K wrote:
> >You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
> >instead of cpan Redis module..
> 
> I didn't notice theres a Redis module included with SpamAssassin.
> It seems like the module is undocumented.
> 
> What does the SA Redis module different from the mainline module?

https://bz.apache.org/SpamAssassin/show_bug.cgi?id=6972

It's made by Mark so quality is assured and it's always bad to rely on
external modules.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Benning, Markus]

On 2016-02-05 16:25, Henrik K wrote:

You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
instead of cpan Redis module..


I didn't notice theres a Redis module included with SpamAssassin.
It seems like the module is undocumented.

What does the SA Redis module different from the mainline module?

Markus

--
https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Henrik K]

On Fri, Feb 05, 2016 at 03:17:21PM +0100, Benning, Markus wrote:
> On 2016-02-04 22:42, Skeffling wrote:
> >Sorry to reply to an old thread, but is it possible for the the
> >RedisAWL
> >plugin to specify a redis password too? (I'm looking to use TxRep
> >with a
> >redis backend...)
> 
> Just released version 1.001:
> 
> commit 4f7138e4823379f7728bcca005611d2a6640cf93
> Author: Markus Benning 
> Date:   Fri Feb 5 15:11:45 2016 +0100
> 
> Additional redis connection parameters
> 
> The followin configuration parameters have been added:
> 
>   * auto_whitelist_redis_password
>   * auto_whitelist_redis_database
>   * auto_whitelist_redis_debug

You should start using the bundled Mail/SpamAssassin/Util/TinyRedis.pm
instead of cpan Redis module..

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Benning, Markus]

On 2016-02-04 22:42, Skeffling wrote:
Sorry to reply to an old thread, but is it possible for the the 
RedisAWL
plugin to specify a redis password too? (I'm looking to use TxRep with 
a

redis backend...)


Just released version 1.001:

commit 4f7138e4823379f7728bcca005611d2a6640cf93
Author: Markus Benning 
Date:   Fri Feb 5 15:11:45 2016 +0100

Additional redis connection parameters

The followin configuration parameters have been added:

  * auto_whitelist_redis_password
  * auto_whitelist_redis_database
  * auto_whitelist_redis_debug

- Markus


--
https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Skeffling]

On 16/07/2015 19:58, Benning, Markus wrote:
> Hi Patrik,
> 
> i just pushed Version 1.002 to github and CPAN:
> 
> -- 
> The following new features have been added:
> 
>   - New option: timing_redis_password allows to specifiy a redis password

Sorry to reply to an old thread, but is it possible for the the RedisAWL
plugin to specify a redis password too? (I'm looking to use TxRep with a
redis backend...)

Thanks.

s.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Martin Gregorie]

On Tue, 2015-09-15 at 08:41 -0700, Ian Zimmerman wrote:
> On 2015-06-09 17:57 +0200, Benning, Markus wrote:
> 
> > RuleTimingRedis - collect SA rule timings in redis
> 
> I'm trying this out.  I have a little annoying problem: the logs
> beginning on line 178 seem to go to stdout or stderr as well as
> syslog.
> The result is that cron sends me email every time spamd is restarted
> (after every rule update).  Do you know how to change that?  I find
> nothing about logging in perldoc Mail::SpamAssassin::Conf.
> 
Assuming your sa_update script is the same as mine (I run Fedora Linux)
The obvious quick fix is to change line 2 in case 0) to read:

   service spamassassin restart;;

and remove lines 1 and 3, which (in my script) are both 'echo' commands
reporting that the update completed and that Spamassassin was
restarted. If you're still seeing the unwanted messages, replace the
remaining line with:

   service spamassassin restart 1>/dev/null 2>/dev.null;; 

will will suppress anything that Spamassassin sends to stdout and
stderr. The sa_update script I have would still report updating errors
and the fact that there was no rules update because these are handled
by case 1, 2 and * and so can't be silenced by this edit.

> I suppose I could just delete those lines from the module :-)  But 
> then I would have extra work when I merge with any new versions you
> have.
> 
Doing what I suggest would be easier than editing SA itself. Keep a
copy of your cron script when its doing what you want so, if an upgrade
slots in a new sa_update script that lets the unwanted text through
again, you can simply slot your modified version back in.


Martin

 

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Ian Zimmerman]

On 2015-06-09 17:57 +0200, Benning, Markus wrote:

> RuleTimingRedis - collect SA rule timings in redis

I'm trying this out.  I have a little annoying problem: the logs
beginning on line 178 seem to go to stdout or stderr as well as syslog.
The result is that cron sends me email every time spamd is restarted
(after every rule update).  Do you know how to change that?  I find
nothing about logging in perldoc Mail::SpamAssassin::Conf.

I suppose I could just delete those lines from the module :-)  But then
I would have extra work when I merge with any new versions you have.

Thanks for your ideas.

-- 
Please *no* private copies of mailing list or newsgroup messages.
Rule 420: All persons more than eight miles high to leave the court.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Patrick Ben Koetter]

Markus,

* Benning, Markus i...@markusbenning.de:
 Hi Patrik,
 
 i just pushed Version 1.002 to github and CPAN:
 
 --
 The following new features have been added:
 
   - New option: timing_redis_password allows to specifiy a redis
 password
 
   - New option: timing_redis_exclude_re excludes rules from timing
 statistics. By default set to '^__' which will exclude all sub-rules
 
   - New option: timing_redis_database allows to select a non-default
 database in redis. (redis SWITCH call)
 
   - New option: timing_redis_bulk_update will queue timing updates
 before sending them to redis and execute them in a bulk via a
 single call to a server-side script. By default this option is set
 to 50 entries. Set to 0 do disable. Requires redis = 2.6.0 and a
 Redis perl = 1.954 module.
 --
 
 I'm currently not using it on a system where the overhead is
 relevant for me, but
 i tried to reduce the number of redis command executed.
 I hope this will reduce the overhead significant.


that's great news. Thanks!

 Feedback and test results welcome.

I will, as soon as I have something to share!

p@rick


 Am 2015-07-15 23:22, schrieb Patrick Ben Koetter:
 Markus,
 
 are you planning to add 'password' and 'database ID' support for redis
 connects to RuleTimingRedis?
 
 What's your experience regarding Timing overhead? My simple tests
 on the
 commandlne show about 1 second overhead when RuleTimingRedis is added:
 
 # Without RuleTimingRedis
 mail# time spamassassin --lint
 
 real0m1.975s
 user0m1.852s
 sys 0m0.116s
 
 # Enable RuleTimingRedis
 mail# vim /etc/mail/spamassassin/init.pre
 
 # With RuleTimingRedis
 mail# time spamassassin --lint
 
 real0m2.828s
 user0m2.128s
 sys 0m0.392s
 
 p@rick
 
 
 
 * Benning, Markus i...@markusbenning.de:
 Hello,
 
 i want to announce the release of the SpamAssassin Plugins:
 
 RedisAWL - redis support for spamassassin AWL/TxRep
 RuleTimingRedis - collect SA rule timings in redis
 
 Both can be downloaded from CPAN or GitHub:
 
 https://metacpan.org/author/BENNING
 
 https://github.com/benningm
 
 Timings gathered with the RuleTimingRedis plugin can be used in
 collectd
 with the Collectd-Plugins-RedisClient module also available from CPAN.
 
  Markus
 
 --
 Markus Benning, https://markusbenning.de/
 
 -- 
 Markus Benning, https://markusbenning.de/

-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Benning, Markus]

Hi Patrik,

i just pushed Version 1.002 to github and CPAN:

--
The following new features have been added:

  - New option: timing_redis_password allows to specifiy a redis 
password


  - New option: timing_redis_exclude_re excludes rules from timing
statistics. By default set to '^__' which will exclude all sub-rules

  - New option: timing_redis_database allows to select a non-default
database in redis. (redis SWITCH call)

  - New option: timing_redis_bulk_update will queue timing updates
before sending them to redis and execute them in a bulk via a
single call to a server-side script. By default this option is set
to 50 entries. Set to 0 do disable. Requires redis = 2.6.0 and a
Redis perl = 1.954 module.
--

I'm currently not using it on a system where the overhead is relevant 
for me, but

i tried to reduce the number of redis command executed.
I hope this will reduce the overhead significant.

Feedback and test results welcome.

Markus

Am 2015-07-15 23:22, schrieb Patrick Ben Koetter:

Markus,

are you planning to add 'password' and 'database ID' support for redis
connects to RuleTimingRedis?

What's your experience regarding Timing overhead? My simple tests on 
the

commandlne show about 1 second overhead when RuleTimingRedis is added:

# Without RuleTimingRedis
mail# time spamassassin --lint

real0m1.975s
user0m1.852s
sys 0m0.116s

# Enable RuleTimingRedis
mail# vim /etc/mail/spamassassin/init.pre

# With RuleTimingRedis
mail# time spamassassin --lint

real0m2.828s
user0m2.128s
sys 0m0.392s

p@rick



* Benning, Markus i...@markusbenning.de:

Hello,

i want to announce the release of the SpamAssassin Plugins:

RedisAWL - redis support for spamassassin AWL/TxRep
RuleTimingRedis - collect SA rule timings in redis

Both can be downloaded from CPAN or GitHub:

https://metacpan.org/author/BENNING

https://github.com/benningm

Timings gathered with the RuleTimingRedis plugin can be used in 
collectd

with the Collectd-Plugins-RedisClient module also available from CPAN.

 Markus

--
Markus Benning, https://markusbenning.de/


--
Markus Benning, https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Patrick Ben Koetter]

Markus,

are you planning to add 'password' and 'database ID' support for redis
connects to RuleTimingRedis?

What's your experience regarding Timing overhead? My simple tests on the
commandlne show about 1 second overhead when RuleTimingRedis is added:

# Without RuleTimingRedis
mail# time spamassassin --lint

real0m1.975s
user0m1.852s
sys 0m0.116s

# Enable RuleTimingRedis
mail# vim /etc/mail/spamassassin/init.pre 

# With RuleTimingRedis
mail# time spamassassin --lint

real0m2.828s
user0m2.128s
sys 0m0.392s

p@rick



* Benning, Markus i...@markusbenning.de:
 Hello,
 
 i want to announce the release of the SpamAssassin Plugins:
 
 RedisAWL - redis support for spamassassin AWL/TxRep
 RuleTimingRedis - collect SA rule timings in redis
 
 Both can be downloaded from CPAN or GitHub:
 
 https://metacpan.org/author/BENNING
 
 https://github.com/benningm
 
 Timings gathered with the RuleTimingRedis plugin can be used in collectd
 with the Collectd-Plugins-RedisClient module also available from CPAN.
 
  Markus
 
 -- 
 Markus Benning, https://markusbenning.de/

-- 
[*] sys4 AG
 
https://sys4.de, +49 (89) 30 90 46 64
Franziskanerstraße 15, 81669 München
 
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263
Vorstand: Patrick Ben Koetter, Marc Schiffbauer
Aufsichtsratsvorsitzender: Florian Kirstein
 

[Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Benning, Markus]

Hello,

i want to announce the release of the SpamAssassin Plugins:

RedisAWL - redis support for spamassassin AWL/TxRep
RuleTimingRedis - collect SA rule timings in redis

Both can be downloaded from CPAN or GitHub:

https://metacpan.org/author/BENNING

https://github.com/benningm

Timings gathered with the RuleTimingRedis plugin can be used in collectd
with the Collectd-Plugins-RedisClient module also available from CPAN.

 Markus

--
Markus Benning, https://markusbenning.de/

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[RW]

On Tue, 09 Jun 2015 21:35:18 +0200
Axb wrote:

 
 suggest you provide a dedicated redisawl.pre including loadplugin
 command
 
 init.pre can be overwritten by an upgrade

That's not supposed to happen.

Re: [Announce] SA-Plugins: RedisAWL, RuleTimingRedis

[Axb]

On 09.06.2015 17:57, Benning, Markus wrote:

Hello,

i want to announce the release of the SpamAssassin Plugins:

RedisAWL - redis support for spamassassin AWL/TxRep
RuleTimingRedis - collect SA rule timings in redis

Both can be downloaded from CPAN or GitHub:

https://metacpan.org/author/BENNING

https://github.com/benningm

Timings gathered with the RuleTimingRedis plugin can be used in collectd
with the Collectd-Plugins-RedisClient module also available from CPAN.

  Markus



interesting

suggest you provide a dedicated redisawl.pre including loadplugin command

init.pre can be overwritten by an upgrade

What kind of traffic have you tested this with?
User count?
Msgs/day/week/month?

Imo, that kind of thing is important with stufff using Redis as you can 
easily max out a machine's memory capacity.