Re: 3.3.0 plans

2009-06-24 Thread Karsten Bräckelmann
On Wed, 2009-06-24 at 23:00 +0100, Justin Mason wrote:
> is there anything else that we should sort out before an alpha is viable?

I believe re-thinking the minimum supported Perl version and related
stuff like screwing MakeMaker would be a *really* good target for a new
$minor release.

How much longer do we want to support Perl 5.6.x? There are comments in
bugzilla from years ago, that there's virtually no system without Perl
5.8.x for years (counting from the comment, not today).

I'd seriously prefer a "drop support for < Perl 5.8.1" in a separate
thread, though.


-- 
char *t="\10pse\0r\0dtu...@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i>=1)||!t[s+h]){ putchar(t[s]);h=m;s=0; }}}



Re: 3.3.0 plans

2009-06-24 Thread Justin Mason
On Wed, Jun 24, 2009 at 21:37, Justin Mason wrote:
> On Wed, Jun 24, 2009 at 21:21, Theo Van Dinter wrote:
>> fwiw, the process used to be:
>> - beta releases to get things stabilized
>> - use a beta release to do mass-check runs
>> - generate scores with mass-check data and submit to svn
>
> I think we may be able to simplify that, now that Daryl's system is
> generating scores weekly...
>
> Also one hard part that we need to do is finish the "no rules in main
> tarball" work item.

which I've just done ;)

is there anything else that we should sort out before an alpha is viable?

--j.

>> - rc releases to get wider testing w/ scores
>> - release after rc appear to work
>>
>>
>>
>> On Wed, Jun 24, 2009 at 2:11 PM, Quanah Gibson-Mount 
>> wrote:
>>> --On Wednesday, June 24, 2009 5:28 PM +0200 Mark Martinec
>>>  wrote:
>>>
>>>
>>>
 On my part I've spent less time lately on SpamAssassin then
 I would like. Getting the amavis release out, preparing for
 a conference, and then there will be a vacation time for me,
 so I won't be of much help until the end of July.

 I wonder if there's anything we could do without too much
 trouble to let more people start using the 3.3 code from CVS.
 I suppose there are still nightly tarball builds? (I haven't
 checked). Perhaps just giving an encouraging hint every now
 and then on a mailing list could be a good start.
>>>
>>> Maybe a public alpha or beta release?  So folks know it isn't official, but
>>> can start testing it out.
>>>
>>> --Quanah
>>>
>>> --
>>>
>>> Quanah Gibson-Mount
>>> Principal Software Engineer
>>> Zimbra, Inc
>>> 
>>> Zimbra ::  the leader in open source messaging and collaboration
>>>
>>
>>
>


[Bug 5752] Let's get rid of the default rules dir and make sa-update mandatory

2009-06-24 Thread bugzilla-daemon
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5752


Justin Mason  changed:

   What|Removed |Added

 Status|ASSIGNED|RESOLVED
 Resolution||FIXED




--- Comment #30 from Justin Mason   2009-06-24 14:55:06 PST ---
closing

-- 
Configure bugmail: 
https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the assignee for the bug.


[Bug 5752] Let's get rid of the default rules dir and make sa-update mandatory

2009-06-24 Thread bugzilla-daemon
https://issues.apache.org/SpamAssassin/show_bug.cgi?id=5752





--- Comment #29 from Justin Mason   2009-06-24 14:54:46 PST ---
(In reply to comment #28)
> We now need to add a way to make a rules-dist tarball; ie. a .tgz containing
> just the rules, in sa-update --install format.  or at least a way to generate
> one of these at release time, and put it up on
> www.apache.org/dist/spamassassin/ alongside the code tarball.


finally done!

: 108...; svn commit -m "bug 5752: get rid of rules in the main distribution
tarball; we now have a new tarball for rules, alongside the main source
tarball, or sa-update can be used to dl the freshest ruleset (more likely)."
Sendingbuild/README
Adding build/update_devel_rules
Sendingbuild/update_stable
Transmitting file data ...
Committed revision 788192 ( 
https://svn.apache.org/viewcvs.cgi?view=rev&rev=788192 ).

maybe we could do an alpha soon, now that this is finally out of the way

-- 
Configure bugmail: 
https://issues.apache.org/SpamAssassin/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are the assignee for the bug.


Re: 3.3.0 plans

2009-06-24 Thread Justin Mason
On Wed, Jun 24, 2009 at 21:21, Theo Van Dinter wrote:
> fwiw, the process used to be:
> - beta releases to get things stabilized
> - use a beta release to do mass-check runs
> - generate scores with mass-check data and submit to svn

I think we may be able to simplify that, now that Daryl's system is
generating scores weekly...

Also one hard part that we need to do is finish the "no rules in main
tarball" work item.

> - rc releases to get wider testing w/ scores
> - release after rc appear to work
>
>
>
> On Wed, Jun 24, 2009 at 2:11 PM, Quanah Gibson-Mount wrote:
>> --On Wednesday, June 24, 2009 5:28 PM +0200 Mark Martinec
>>  wrote:
>>
>>
>>
>>> On my part I've spent less time lately on SpamAssassin then
>>> I would like. Getting the amavis release out, preparing for
>>> a conference, and then there will be a vacation time for me,
>>> so I won't be of much help until the end of July.
>>>
>>> I wonder if there's anything we could do without too much
>>> trouble to let more people start using the 3.3 code from CVS.
>>> I suppose there are still nightly tarball builds? (I haven't
>>> checked). Perhaps just giving an encouraging hint every now
>>> and then on a mailing list could be a good start.
>>
>> Maybe a public alpha or beta release?  So folks know it isn't official, but
>> can start testing it out.
>>
>> --Quanah
>>
>> --
>>
>> Quanah Gibson-Mount
>> Principal Software Engineer
>> Zimbra, Inc
>> 
>> Zimbra ::  the leader in open source messaging and collaboration
>>
>
>


Re: 3.3.0 plans

2009-06-24 Thread Justin Mason
On Wed, Jun 24, 2009 at 19:11, Quanah Gibson-Mount wrote:
> --On Wednesday, June 24, 2009 5:28 PM +0200 Mark Martinec
>  wrote:
>
>> On my part I've spent less time lately on SpamAssassin then
>> I would like. Getting the amavis release out, preparing for
>> a conference, and then there will be a vacation time for me,
>> so I won't be of much help until the end of July.
>>
>> I wonder if there's anything we could do without too much
>> trouble to let more people start using the 3.3 code from CVS.
>> I suppose there are still nightly tarball builds? (I haven't
>> checked). Perhaps just giving an encouraging hint every now
>> and then on a mailing list could be a good start.
>
> Maybe a public alpha or beta release?  So folks know it isn't official, but
> can start testing it out.

A public alpha might be a good option.  The main stuff we need to do
after that would be procedural changes and score generation.

--j.

> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> 
> Zimbra ::  the leader in open source messaging and collaboration
>
>


Re: 3.3.0 plans

2009-06-24 Thread Theo Van Dinter
fwiw, the process used to be:
- beta releases to get things stabilized
- use a beta release to do mass-check runs
- generate scores with mass-check data and submit to svn
- rc releases to get wider testing w/ scores
- release after rc appear to work



On Wed, Jun 24, 2009 at 2:11 PM, Quanah Gibson-Mount wrote:
> --On Wednesday, June 24, 2009 5:28 PM +0200 Mark Martinec
>  wrote:
>
>
>
>> On my part I've spent less time lately on SpamAssassin then
>> I would like. Getting the amavis release out, preparing for
>> a conference, and then there will be a vacation time for me,
>> so I won't be of much help until the end of July.
>>
>> I wonder if there's anything we could do without too much
>> trouble to let more people start using the 3.3 code from CVS.
>> I suppose there are still nightly tarball builds? (I haven't
>> checked). Perhaps just giving an encouraging hint every now
>> and then on a mailing list could be a good start.
>
> Maybe a public alpha or beta release?  So folks know it isn't official, but
> can start testing it out.
>
> --Quanah
>
> --
>
> Quanah Gibson-Mount
> Principal Software Engineer
> Zimbra, Inc
> 
> Zimbra ::  the leader in open source messaging and collaboration
>


Re: 3.3.0 plans

2009-06-24 Thread Quanah Gibson-Mount
--On Wednesday, June 24, 2009 5:28 PM +0200 Mark Martinec 
 wrote:





On my part I've spent less time lately on SpamAssassin then
I would like. Getting the amavis release out, preparing for
a conference, and then there will be a vacation time for me,
so I won't be of much help until the end of July.

I wonder if there's anything we could do without too much
trouble to let more people start using the 3.3 code from CVS.
I suppose there are still nightly tarball builds? (I haven't
checked). Perhaps just giving an encouraging hint every now
and then on a mailing list could be a good start.


Maybe a public alpha or beta release?  So folks know it isn't official, but 
can start testing it out.


--Quanah

--

Quanah Gibson-Mount
Principal Software Engineer
Zimbra, Inc

Zimbra ::  the leader in open source messaging and collaboration


Re: 3.3.0 plans

2009-06-24 Thread Mark Martinec
Sorry for my late response, and bringing the topic back to the list
(with some quotes omitted).

Warren Togami writes:
> Might it be feasible to get 3.3.0 out sometime during the Fedora 12 dev
> cycle?  It seems there is maybe 2-3 months.  A fresh spamassassin making
> this release might possibly be particularly more important than other
> releases for unnamed reasons...

Justin writes:
> It would be quite possible to do this, given a sufficiently-dedicated
> release guy pushing it.  It takes about 2 months to get through
> mass-checks, generate good scores, etc. Unfortunately, I'm out for this
> job, as child #2 is about to be born any day now ;)
>
> BTW in the short term, if ppl want to help 3.3.0 release plans, I suggest
> trying out the SVN trunk code as "dogfood".  I've been doing this for the
> last couple of years with very good results; I can't recall the last time
> I had to restart my spamd, for example, or spotted an FP...

I've also been running a fresh 3.3 in production at our site for more
than a year now, mostly because it provides bug fixes and some new
features or workarounds that I need. For example with a Perl upgrade
to 5.8.9 (and later to 5.10.0) on FreeBSD ports the old 3.2.5 just
couldn't survive any longer without crashing due to exceeded stack
size while compiling our set of rules (a perl bug).

I can confirm that 3.3 fares at least as well as 3.2.5, and in
many ways better. I do monitor for possible glitches, but there
just aren't any (new ones that is, that would not also be in 3.2.5).
I may also add that the more recent versions of amavisd-new are
happier with 3.3 than with 3.2.5, as this enables some nice SA
functionality (like SA timing breakdown reports, DKIM checking
of long mail messages).

The trouble with only a handful of sites running the new code is
that not all features are exercised, and not all setups tested.

Then there is a concern that some of the fixes for old bug reports
would still be nice to get into a 3.3 before a release, and some
nice features added/finished. Given the current progress trends
this is unfortunately just stretching time to a release, rising
a feeling that "it just isn't ripe yet". We might consider just
setting our goal/expectations closer to ground, and come out with
what we have.

> I'm pretty sure this won't be achievable in that timeframe if I'm
> driving the release process [...] while I'm busy running after babies ;)
> guys, I can totally provide advice/guidance as to how to go about
> this, if you feel up to it.
>
> basically this wiki page describes the basic idea of how to put
> together a new x.y.0 release:
> http://wiki.apache.org/spamassassin/ReleasePolicy
>
> the biggest task is working through this:
> http://wiki.apache.org/spamassassin/RescoreMassCheck

On my part I've spent less time lately on SpamAssassin then
I would like. Getting the amavis release out, preparing for
a conference, and then there will be a vacation time for me,
so I won't be of much help until the end of July.

I wonder if there's anything we could do without too much
trouble to let more people start using the 3.3 code from CVS.
I suppose there are still nightly tarball builds? (I haven't
checked). Perhaps just giving an encouraging hint every now
and then on a mailing list could be a good start.

  Mark


[auto] bad sandbox rules report

2009-06-24 Thread Rules Report Cron
HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/3-days-ago?xml=1

Bad performing rules, from the past 3 night's mass-checks.

(Note: 'net' rules will be listed as 'no hits' unless you set 'tflags net'.
This also applies for meta rules which use 'net' rules.)

plugin: failed to create instance of plugin 
Mail::SpamAssassin::Plugin::Sandbox::hstern: Can't locate object method "new" 
via package "Mail::SpamAssassin::Plugin::Sandbox::hstern" (perhaps you forgot 
to load "Mail::SpamAssassin::Plugin::Sandbox::hstern"?) at (eval 271) line 1.

plugin: failed to create instance of plugin 
Mail::SpamAssassin::Plugin::FreeMail: Can't locate object method "new" via 
package "Mail::SpamAssassin::Plugin::FreeMail" (perhaps you forgot to load 
"Mail::SpamAssassin::Plugin::FreeMail"?) at (eval 272) line 1.

Use of uninitialized value in concatenation (.) or string at 
masses/rule-qa/list-bad-rules line 181.
demoting : config: found endif without matching conditional at 
masses/rule-qa/list-bad-rules line 181.
Use of uninitialized value in hash element at masses/rule-qa/list-bad-rules 
line 183.

rulesrc/sandbox/mmartinec/25_yg.cf (17 rules, 17 bad):

  NOTVALID_GMAIL:  bad, avg S/O=0.52 avg Spam%=0.44 avg Ham%=0.40
  NOTVALID_PAY:  bad, avg S/O=0.17 avg Spam%=0.05 avg Ham%=0.24
  NOTVALID_YAHOO:  bad, avg S/O=0.55 avg Spam%=0.85 avg Ham%=0.69
  __AUTH_EBAY:  bad, avg S/O=0.02 avg Spam%=0.00 avg Ham%=0.20
  # used in: NOTVALID_PAY
  __AUTH_GMAIL:  bad, avg S/O=0.04 avg Spam%=0.44 avg Ham%=11.44
  # used in: NOTVALID_GMAIL
  __AUTH_PAYPAL:  bad, avg S/O=0.57 avg Spam%=0.04 avg Ham%=0.03
  # used in: NOTVALID_PAY
  __AUTH_YAHOO:  bad, avg S/O=0.16 avg Spam%=0.85 avg Ham%=4.50
  # used in: NOTVALID_YAHOO
  __AUTH_YAHOO1:  bad, avg S/O=0.16 avg Spam%=0.51 avg Ham%=2.59
  # used in: NOTVALID_YAHOO __AUTH_YAHOO
  __AUTH_YAHOO2:  bad, avg S/O=0.71 avg Spam%=0.09 avg Ham%=0.04
  # used in: NOTVALID_YAHOO __AUTH_YAHOO
  __AUTH_YAHOO3:  bad, avg S/O=0.13 avg Spam%=0.13 avg Ham%=0.90
  # used in: NOTVALID_YAHOO __AUTH_YAHOO
  __AUTH_YAHOO4:  bad, avg S/O=0.11 avg Spam%=0.12 avg Ham%=0.97
  # used in: NOTVALID_YAHOO __AUTH_YAHOO
  __ML1:  bad, avg S/O=0.00 avg Spam%=0.01 avg Ham%=59.83
  # used in: NOTVALID_GMAIL NOTVALID_YAHOO
  __ML2:  bad, avg S/O=0.00 avg Spam%=0.04 avg Ham%=72.32
  # used in: NOTVALID_GMAIL NOTVALID_YAHOO
  __ML3:  bad, avg S/O=0.00 avg Spam%=0.00 avg Ham%=63.44
  # used in: NOTVALID_GMAIL NOTVALID_YAHOO
  __ML4:  bad, avg S/O=0.00 avg Spam%=0.00 avg Ham%=39.04
  # used in: NOTVALID_GMAIL NOTVALID_YAHOO
  __ML5:  bad, avg S/O=0.00 avg Spam%=0.07 avg Ham%=24.03
  # used in: NOTVALID_GMAIL NOTVALID_YAHOO
  __VIA_ML:  bad, avg S/O=0.00 avg Spam%=0.11 avg Ham%=74.47
  # used in: NOTVALID_GMAIL NOTVALID_YAHOO

rulesrc/sandbox/mmartinec/20_misc.cf (12 rules, 2 bad):

  LONGLINE:  bad, avg S/O=0.03 avg Spam%=0.02 avg Ham%=0.64
  MSGID_NOFQDN2:  bad, avg S/O=0.72 avg Spam%=25.36 avg Ham%=9.88

rulesrc/sandbox/maddoc/99_doc_test.cf (37 rules, 15 bad):

  FSL_BLOGSPOT_ABUSE:  bad, avg S/O=0.20 avg Spam%=0.23 avg Ham%=0.91
  FSL_FREEMAIL_1:  bad, avg S/O=0.49 avg Spam%=1.87 avg Ham%=1.95
  FSL_FREEMAIL_2:  bad, avg S/O=0.40 avg Spam%=0.77 avg Ham%=1.16
  FSL_GD2_URI:  no hits of target type
  FSL_HAS_TINYURL:  bad, avg S/O=0.01 avg Spam%=0.00 avg Ham%=0.32
  FSL_HELO_HOME:  bad, avg S/O=0.77 avg Spam%=0.51 avg Ham%=0.15
  FSL_HELO_NON_FQDN_2:  bad, avg S/O=0.47 avg Spam%=86.64 avg Ham%=99.12
  FSL_SINGLE_URI:  bad, avg S/O=0.58 avg Spam%=35.54 avg Ham%=26.10
  FSL_SPAMWARE_STRING_1:  bad, avg S/O=0.41 avg Spam%=0.05 avg Ham%=0.08
  FSL_UA:  bad, avg S/O=0.72 avg Spam%=0.20 avg Ham%=0.08
  __ANY_HTTP_URI:  bad, avg S/O=0.54 avg Spam%=89.42 avg Ham%=77.13
  # used in: FSL_SINGLE_URI
  __FROM_FREEMAIL:  bad, avg S/O=0.11 avg Spam%=2.19 avg Ham%=17.62
  # used in: FSL_FREEMAIL_2
  __FSL_UA_2:  bad, avg S/O=0.72 avg Spam%=0.20 avg Ham%=0.08
  # used in: FSL_UA
  __HAS_REPLY_TO:  bad, avg S/O=0.19 avg Spam%=10.95 avg Ham%=47.94
  # used in: FSL_FREEMAIL_1 FSL_FREEMAIL_2
  __REPLY_FREEMAIL:  bad, avg S/O=0.49 avg Spam%=1.87 avg Ham%=1.95
  # used in: FSL_FREEMAIL_1 FSL_FREEMAIL_2

rulesrc/sandbox/kmcgrail/70_phishing.cf (1 rules, 1 bad):

  KAM_PHISH1:  no hits of target type

rulesrc/sandbox/kmcgrail/70_mx.cf (4 rules, 2 bad):

  KAM_MX:  bad, avg S/O=0.35 avg Spam%=2.04 avg Ham%=3.82
  __KAM_MX3:  bad, avg S/O=0.35 avg Spam%=2.03 avg Ham%=3.82
  # used in: KAM_MX

rulesrc/sandbox/kmcgrail/20_test.cf (19 rules, 2 bad):

  KAM_REPLACE:  bad, avg S/O=0.02 avg Spam%=0.01 avg Ham%=0.25
  __KAM_REP2:  bad, avg S/O=0.02 avg Spam%=0.01 avg Ham%=0.25
  # used in: KAM_REPLACE

rulesrc/sandbox/jm/80_sane.cf (61 rules, 4 bad):

  SANE_3ca2a5874272d53ce2ad613e09b71fce:  bad, avg S/O=0.37 avg Spam%=0.01 avg 
Ham%=0.02
  SANE_d0d2b0f6373bf91253d66dd74c59