sa-updates not happening (Re: Update Mirror Issues)

2011-03-20 Thread Darxus
On 01/27, Daryl C. W. O'Shea wrote: Not quite sure why 3.3.0 would be different from 3.3.1+2 would be different, but the reason we haven't had any stable branch rules published in a while is that we haven't had enough *recent* spam submitted. Last nights cron job says: HAM: 188008 (15

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-03 Thread João Gouveia
- Warren Togami Jr. wtog...@gmail.com wrote: On 2/2/2011 5:25 PM, Karsten Bräckelmann wrote: Spam to live accounts strongly preferred, human reviewed by trained monkeys. Emphasis on trained. ;) Some crap like backscatter should be filtered from the trap data, if possible, and trap

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-03 Thread Warren Togami Jr.
On 2/2/2011 11:52 PM, João Gouveia wrote: Thanks for the clear specs Warren, that helps ;-) We shall try to do it like that. I still need to setup a proper environment for this. Hopefully on this next weekend. Saw my previous post about spamassassin headers and the importance of reuse?

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-03 Thread Justin Mason
2011/2/3 Warren Togami Jr. wtog...@gmail.com: On 2/2/2011 11:52 PM, João Gouveia wrote: Thanks for the clear specs Warren, that helps ;-) We shall try to do it like that. I still need to setup a proper environment for this. Hopefully on this next weekend. Saw my previous post about

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-03 Thread Warren Togami Jr.
On 2/3/2011 1:02 AM, Justin Mason wrote: I think there's probably quite a few existing mass-check submitters who are in the same boat, actually... --j. And that is a serious problem. We have a similar problem with accurately measuring network rules that are not yet part of the production

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-02 Thread Justin Mason
2011/2/2 Warren Togami Jr. wtog...@gmail.com: On 2/1/2011 1:02 PM, Karsten Bräckelmann wrote: Yikes indeed. Maybe Joao should answer these himself... Given the numbers, is that purely trap driven? Is there a legion human users manually verifying the spam? What exactly does filter

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-02 Thread João Gouveia
- Karsten Bräckelmann guent...@rudersport.de wrote: SPAM: 51330 (15 required) Joao Gouveia will soon be requesting an account to join the nightly masscheck. He has a significant quantity of spam, and hopefully much of it is European language so it should add to our

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-02 Thread Karsten Bräckelmann
On Thu, 2011-02-03 at 02:40 +, João Gouveia wrote: - Karsten Bräckelmann guent...@rudersport.de wrote: Given the numbers, is that purely trap driven? Is there a legion human users manually verifying the spam? Can't really go into much detail (unrelated to Spamassassin), but there

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-02 Thread Warren Togami Jr.
On 2/2/2011 5:25 PM, Karsten Bräckelmann wrote: Spam to live accounts strongly preferred, human reviewed by trained monkeys. Emphasis on trained. ;) Some crap like backscatter should be filtered from the trap data, if possible, and trap volume kept lower -- best done by random sampling, rather

Re: Update Mirror Issues

2011-02-01 Thread John Hardin
On Mon, 31 Jan 2011, Warren Togami Jr. wrote: On 01/26/2011 06:48 PM, Daryl C. W. O'Shea wrote: SPAM: 51330 (15 required) Joao Gouveia will soon be requesting an account to join the nightly masscheck. He has a significant quantity of spam, and hopefully much of it is European

Re: Update Mirror Issues

2011-02-01 Thread Warren Togami Jr.
On 2/1/2011 5:44 AM, John Hardin wrote: On Mon, 31 Jan 2011, Warren Togami Jr. wrote: On 01/26/2011 06:48 PM, Daryl C. W. O'Shea wrote: SPAM: 51330 (15 required) Joao Gouveia will soon be requesting an account to join the nightly masscheck. He has a significant quantity of spam, and

Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-01 Thread Karsten Bräckelmann
SPAM: 51330 (15 required) Joao Gouveia will soon be requesting an account to join the nightly masscheck. He has a significant quantity of spam, and hopefully much of it is European language so it should add to our diversity. I wonder how scoring will be affected if his

Re: Mass-check Corpora (once was: Re: Update Mirror Issues)

2011-02-01 Thread Warren Togami Jr.
On 2/1/2011 1:02 PM, Karsten Bräckelmann wrote: Yikes indeed. Maybe Joao should answer these himself... Given the numbers, is that purely trap driven? Is there a legion human users manually verifying the spam? What exactly does filter duplicates mean? If that includes identical payload sent

Re: Update Mirror Issues

2011-01-31 Thread Warren Togami Jr.
On 01/26/2011 06:48 PM, Daryl C. W. O'Shea wrote: Not quite sure why 3.3.0 would be different from 3.3.1+2 would be different, but the reason we haven't had any stable branch rules published in a while is that we haven't had enough *recent* spam submitted. Last nights cron job says: HAM: 188008

Re: Update Mirror Issues

2011-01-31 Thread Kevin A. McGrail
On 1/31/2011 11:40 PM, Warren Togami Jr. wrote: On 01/26/2011 06:48 PM, Daryl C. W. O'Shea wrote: Not quite sure why 3.3.0 would be different from 3.3.1+2 would be different, but the reason we haven't had any stable branch rules published in a while is that we haven't had enough *recent* spam

Re: Update Mirror Issues

2011-01-28 Thread Justin Mason
ah, my apologies -- I'd forgotten about that. As the bug notes, we never completed instructions to reliably push an update, so I'm not really sure what the correct approach is --j. On Thu, Jan 27, 2011 at 12:02, Warren Togami Jr. wtog...@gmail.com wrote: On 01/19/2011 11:58 AM, Justin

Re: Update Mirror Issues

2011-01-27 Thread Warren Togami Jr.
On 1/26/2011 6:48 PM, Daryl C. W. O'Shea wrote: On 26/01/2011 10:12 PM, Kevin A. McGrail wrote: On 1/26/2011 5:39 PM, Karsten Bräckelmann wrote: Just came up on the users list. Escalating. ;) The facts: 1.3.3.updates.spamassassin.org descriptive text 1052462 2.3.3.updates.spamassassin.org

Re: Update Mirror Issues

2011-01-27 Thread Warren Togami Jr.
On 1/26/2011 7:29 PM, Daryl C. W. O'Shea wrote: OK, found the cause. Somebody broke it by manually changing the 3.3.0 update version in DNS to point at 1061118 which is a 3.4.0 version (and not necessarily compatible with 3.3.0!). This is seriously bad. I won't point fingers, but here's the

Re: Update Mirror Issues

2011-01-27 Thread Warren Togami Jr.
On 01/19/2011 11:58 AM, Justin Mason wrote: ssh spamassassin.zones.apache.org cd /home/updatesd/svn/spamassassin/build/mkupdates [svn up appropriately] sudo -u updatesd ./update-rules-3.3 3.3 see build/README for full details. These were JM's

Re: Update Mirror Issues

2011-01-27 Thread Daryl C. W. O'Shea
On 27/01/2011 7:02 AM, Warren Togami Jr. wrote: On 01/19/2011 11:58 AM, Justin Mason wrote: ssh spamassassin.zones.apache.org cd /home/updatesd/svn/spamassassin/build/mkupdates [svn up appropriately] sudo -u updatesd ./update-rules-3.3 3.3 see build/README for full details. These

Re: Update Mirror Issues

2011-01-27 Thread Daryl C. W. O'Shea
On 27/01/2011 6:14 AM, Warren Togami Jr. wrote: On 1/26/2011 6:48 PM, Daryl C. W. O'Shea wrote: On 26/01/2011 10:12 PM, Kevin A. McGrail wrote: On 1/26/2011 5:39 PM, Karsten Bräckelmann wrote: Just came up on the users list. Escalating. ;) The facts: 1.3.3.updates.spamassassin.org

Re: Update Mirror Issues

2011-01-27 Thread Warren Togami Jr.
On 01/27/2011 12:40 PM, Daryl C. W. O'Shea wrote: HAM: 188008 (15 required) SPAM: 51330 (15 required) Insufficient spam corpus to generate scores; aborting. How recent is recent? Ham: 72 months Spam: 2 months Daryl Would it be too harmful to expand it to 3-4 months temporarily?

Update Mirror Issues

2011-01-26 Thread Karsten Bräckelmann
Just came up on the users list. Escalating. ;) The facts: 1.3.3.updates.spamassassin.org descriptive text 1052462 2.3.3.updates.spamassassin.org descriptive text 1052462 Rule update tarball available on mirrors. 4 weeks old revision from trunk. 0.3.3.updates.spamassassin.org descriptive

Re: Update Mirror Issues

2011-01-26 Thread Kevin A. McGrail
On 1/26/2011 5:39 PM, Karsten Bräckelmann wrote: Just came up on the users list. Escalating. ;) The facts: 1.3.3.updates.spamassassin.org descriptive text 1052462 2.3.3.updates.spamassassin.org descriptive text 1052462 Rule update tarball available on mirrors. 4 weeks old revision from

Re: Update Mirror Issues

2011-01-26 Thread Daryl C. W. O'Shea
On 26/01/2011 10:12 PM, Kevin A. McGrail wrote: On 1/26/2011 5:39 PM, Karsten Bräckelmann wrote: Just came up on the users list. Escalating. ;) The facts: 1.3.3.updates.spamassassin.org descriptive text 1052462 2.3.3.updates.spamassassin.org descriptive text 1052462 Rule update tarball

Re: Update Mirror Issues

2011-01-26 Thread Daryl C. W. O'Shea
On 26/01/2011 11:48 PM, Daryl C. W. O'Shea wrote: On 26/01/2011 10:12 PM, Kevin A. McGrail wrote: On 1/26/2011 5:39 PM, Karsten Bräckelmann wrote: Just came up on the users list. Escalating. ;) The facts: 1.3.3.updates.spamassassin.org descriptive text 1052462 2.3.3.updates.spamassassin.org

Re: Update Mirror Issues

2011-01-26 Thread Daryl C. W. O'Shea
Corrected manual procedure... the at job needs to be run as updatesd: On 27/01/2011 12:29 AM, Daryl C. W. O'Shea wrote: Going forward... we, probably me, need to get an automated way to push some sort of emergency rule update. The current manually steps would be: - un-tar an existing STABLE