Re: Trying Bayes / Redis
On 12/15/2015 10:57 PM, Marc Perkel wrote: This Bayes Redis works GREAT. For years I've been trying to get bayes to work and now finally IT WORKS good news for once... just watch memory usage... :)
Re: Trying Bayes / Redis
This Bayes Redis works GREAT. For years I've been trying to get bayes to work and now finally IT WORKS -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
On December 13, 2015 1:16:17 AM Marc Perkel wrote: Because I'd have to upgrade 50 servers for consistency and if I do that I'll probably try something other than centos. okay, it just not how i would solve +1 server farms in gentoo, here i would emerge --buildpkgonly on master, and then emerge --usepkgonly on all 50, no need to use distcc, its just for me another world back to my problems with fuglu ebuild, its added to maintainer wanted in gentoo now if others like to help
Re: Trying Bayes / Redis
On 12/12/15 15:28, Benny Pedersen wrote: On December 12, 2015 5:49:27 PM Marc Perkel wrote: I can put redis in thew yum exception list so much precompiled problems, why not upgrade to centos 7 ? Because I'd have to upgrade 50 servers for consistency and if I do that I'll probably try something other than centos. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
Am 13.12.2015 um 00:28 schrieb Benny Pedersen: On December 12, 2015 5:49:27 PM Marc Perkel wrote: I can put redis in thew yum exception list so much precompiled problems blablub nonsense as usual why not upgrade to centos 7 ? because it's the purpose of a LTS not have major updates for a ton of packages all the time? signature.asc Description: OpenPGP digital signature
Re: Trying Bayes / Redis
On December 12, 2015 5:49:27 PM Marc Perkel wrote: I can put redis in thew yum exception list so much precompiled problems, why not upgrade to centos 7 ?
Re: Trying Bayes / Redis
Am 12.12.2015 um 18:57 schrieb Marc Perkel: On 12/12/15 09:36, Martin Gregorie wrote: On Sat, 2015-12-12 at 08:49 -0800, Marc Perkel wrote: I can put redis in thew yum exception list Or, you can install your self-built version in /usr/local/bin and adjust $PATH so it preceeds /usr and /usr/bin. This will protect your version from yum or dnf updates. Then keep an eye on subsequent updates and remove your version if/when yum or dnf delivers a more recent version. Actually I did rpm -e --justdb to take it out so it doesn't get updated as long there is nothing with a dependency to it installed and get a update later :-) signature.asc Description: OpenPGP digital signature
Re: Trying Bayes / Redis
On 12/12/15 09:36, Martin Gregorie wrote: On Sat, 2015-12-12 at 08:49 -0800, Marc Perkel wrote: I can put redis in thew yum exception list Or, you can install your self-built version in /usr/local/bin and adjust $PATH so it preceeds /usr and /usr/bin. This will protect your version from yum or dnf updates. Then keep an eye on subsequent updates and remove your version if/when yum or dnf delivers a more recent version. Martin Actually I did rpm -e --justdb to take it out so it doesn't get updated. BTW - this is working GREAT. I'm thinking about learning redis and doing interesting things with it. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
On Sat, 2015-12-12 at 08:49 -0800, Marc Perkel wrote: > I can put redis in thew yum exception list > Or, you can install your self-built version in /usr/local/bin and adjust $PATH so it preceeds /usr and /usr/bin. This will protect your version from yum or dnf updates. Then keep an eye on subsequent updates and remove your version if/when yum or dnf delivers a more recent version. Martin
Re: Trying Bayes / Redis
Am 12.12.2015 um 18:24 schrieb Marc Perkel: On 12/12/15 02:38, Axb wrote: On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade... What does LUA mean? https://en.wikipedia.org/wiki/Lua_%28programming_language%29 heavily used for many things "embeede" in mod_security, apache trafficserver as a lightweight scripting language signature.asc Description: OpenPGP digital signature
Re: Trying Bayes / Redis
On 12/12/15 02:38, Axb wrote: On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade... What does LUA mean? -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
sa-learn --dump magic 0.000 0 3 0 non-token data: bayes db version 0.000 0 69569 0 non-token data: nspam 0.000 0 88747 0 non-token data: nham 0.000 0 0 0 non-token data: ntokens 0.000 0 0 0 non-token data: oldest atime 0.000 0 0 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count # Server redis_version:3.0.5 redis_git_sha1: redis_git_dirty:0 redis_build_id:a0e516305b2572d8 redis_mode:standalone os:Linux 2.6.32-042stab112.15 x86_64 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.7 process_id:2085 run_id:f2436709b788f7a9b6a043f6ed90a49512c049d2 tcp_port:6379 uptime_in_seconds:685 uptime_in_days:0 hz:10 lru_clock:7100234 config_file:/etc/redis.conf # Clients connected_clients:550 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 # Memory used_memory:905452784 used_memory_human:863.51M used_memory_rss:924852224 used_memory_peak:964409608 used_memory_peak_human:919.73M used_memory_lua:177152 mem_fragmentation_ratio:1.02 mem_allocator:jemalloc-3.6.0 # Persistence loading:0 rdb_changes_since_last_save:98170 rdb_bgsave_in_progress:1 rdb_last_save_time:1449940748 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:7 rdb_current_bgsave_time_sec:1 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok # Stats total_connections_received:3806 total_commands_processed:1302390 instantaneous_ops_per_sec:2133 total_net_input_bytes:52637812 total_net_output_bytes:15806201 instantaneous_input_kbps:80.48 instantaneous_output_kbps:23.47 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 evicted_keys:0 keyspace_hits:194094 keyspace_misses:65528 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:44672 migrate_cached_sockets:0 # Replication role:master connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 # CPU used_cpu_sys:5.97 used_cpu_user:34.09 used_cpu_sys_children:4.85 used_cpu_user_children:57.29 # Cluster cluster_enabled:0 # Keyspace db0:keys=3,expires=0,avg_ttl=0 db2:keys=9277218,expires=4,avg_ttl=2546306553
Re: Trying Bayes / Redis
On 12/12/15 08:47, Reindl Harald wrote: Am 12.12.2015 um 17:44 schrieb Marc Perkel: On 12/12/15 02:38, Axb wrote: On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade... I'm running Centos 6. I just downloaded and compiled the latest. Can I just copy over the executables and keep my data or do I need to start over? if it works with the old data you can keep it, otherwise as said "probably need to nuke your DB dump" but don't do make && make install on producton machines sooner or later your build will get overwritten by a yum-update with the old version + patches because rpm don't know anything about your override https://wiki.centos.org/HowTos/SetupRpmBuildEnvironment Worked with the old data. -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
On 12/12/15 08:47, Reindl Harald wrote: Am 12.12.2015 um 17:44 schrieb Marc Perkel: On 12/12/15 02:38, Axb wrote: On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade... I'm running Centos 6. I just downloaded and compiled the latest. Can I just copy over the executables and keep my data or do I need to start over? if it works with the old data you can keep it, otherwise as said "probably need to nuke your DB dump" but don't do make && make install on producton machines sooner or later your build will get overwritten by a yum-update with the old version + patches because rpm don't know anything about your override https://wiki.centos.org/HowTos/SetupRpmBuildEnvironment I can put redis in thew yum exception list -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
Am 12.12.2015 um 17:44 schrieb Marc Perkel: On 12/12/15 02:38, Axb wrote: On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade... I'm running Centos 6. I just downloaded and compiled the latest. Can I just copy over the executables and keep my data or do I need to start over? if it works with the old data you can keep it, otherwise as said "probably need to nuke your DB dump" but don't do make && make install on producton machines sooner or later your build will get overwritten by a yum-update with the old version + patches because rpm don't know anything about your override https://wiki.centos.org/HowTos/SetupRpmBuildEnvironment signature.asc Description: OpenPGP digital signature
Re: Trying Bayes / Redis
On 12/12/15 02:38, Axb wrote: On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade... I'm running Centos 6. I just downloaded and compiled the latest. Can I just copy over the executables and keep my data or do I need to start over? -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
On 12/12/2015 12:28 AM, Marc Perkel wrote: redis_version:2.4.10 You're using an ancient Redis version... SA makes use of LUA support which was added in 2.6.0 You definitely need to upgrde to 3.x and you'll probably need to nuke your DB dump before the upgrade...
Re: Trying Bayes / Redis
On 2015-12-11 14:29 -0800, Marc Perkel wrote: > Anyone using this rule timing plugin? Having trouble getting it to > work. Just wondering if it's worth it? > > Mail::SpamAssassin::Plugin::RuleTimingRedis I use it and I have no trouble now. But I remember I had to disable the LUA scripting stuff when I set it up, it wouldn't work even though my Redis version should be recent enough to support it. -- Please *no* private copies of mailing list or newsgroup messages. Rule 420: All persons more than eight miles high to leave the court.
Re: Trying Bayes / Redis
On 12/11/15 14:59, Axb wrote: On 12/11/2015 11:55 PM, Marc Perkel wrote: So far so good. Thanks for your help. It looks like it's actually working. Have about 10k spam/ham tokens so far. sa-learn --dump magic should show if it's "alive" For safety, don't take your eyes off memory usage and play with: bayes_token_ttl bayes_seen_ttl I've set bayes_seen_ttl 1 Curious what your "redis-cli info" looks like after a while of running... adn if you're happy with speed and results... keep us posted... Axb 0.000 0 3 0 non-token data: bayes db version 0.000 0 14814 0 non-token data: nspam 0.000 0 11933 0 non-token data: nham 0.000 0 0 0 non-token data: ntokens 0.000 0 0 0 non-token data: oldest atime 0.000 0 0 0 non-token data: newest atime 0.000 0 0 0 non-token data: last journal sync atime 0.000 0 0 0 non-token data: last expiry atime 0.000 0 0 0 non-token data: last expire atime delta 0.000 0 0 0 non-token data: last expire reduction count redis_version:2.4.10 redis_git_sha1: redis_git_dirty:0 arch_bits:64 multiplexing_api:epoll gcc_version:4.4.6 process_id:472 uptime_in_seconds:4846 uptime_in_days:0 lru_clock:284149 used_cpu_sys:29.75 used_cpu_user:91.39 used_cpu_sys_children:4.31 used_cpu_user_children:49.34 connected_clients:671 connected_slaves:0 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 used_memory:185037872 used_memory_human:176.47M used_memory_rss:182960128 used_memory_peak:193339376 used_memory_peak_human:184.38M mem_fragmentation_ratio:0.99 mem_allocator:jemalloc-2.2.5 loading:0 aof_enabled:0 changes_since_last_save:42332 bgsave_in_progress:0 last_save_time:1449876339 bgrewriteaof_in_progress:0 total_connections_received:110460 total_commands_processed:9275568 expired_keys:0 evicted_keys:0 keyspace_hits:1821764 keyspace_misses:793471 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:16204 vm_enabled:0 role:master db2:keys=2012232,expires=4 -- Marc Perkel - Sales/Support supp...@junkemailfilter.com http://www.junkemailfilter.com Junk Email Filter dot com 415-992-3400
Re: Trying Bayes / Redis
On 12/11/2015 11:55 PM, Marc Perkel wrote: So far so good. Thanks for your help. It looks like it's actually working. Have about 10k spam/ham tokens so far. sa-learn --dump magic should show if it's "alive" For safety, don't take your eyes off memory usage and play with: bayes_token_ttl bayes_seen_ttl I've set bayes_seen_ttl 1 Curious what your "redis-cli info" looks like after a while of running... adn if you're happy with speed and results... keep us posted... Axb
Re: Trying Bayes / Redis
So far so good. Thanks for your help. It looks like it's actually working. Have about 10k spam/ham tokens so far.