updateDNS.sh on sa-vm1.apache.org - DNS updates disabled

[noreply]

3.3.3.updates (TXT) -> \"1814340\"

File /usr/local/bin/updateDNS.disabled exists, not updating DNS.

Re: mailspike in 72_scores

[Kevin A. McGrail]

You should rant about rspamd on list so you can shame moee into working on it 
:-)
Regards,
KAM

On November 5, 2017 12:21:03 PM EST, Dave Jones  wrote:
>On 11/05/2017 10:26 AM, Merijn van den Kroonenberg wrote:
>> Note to self: investigate why in do-nightly-rescore-example.sh is
>called
>> like this:
>> merge-scoresets $SCORESET
>> With a $SCORESET param. It will cause all rules which are not in that
>> scores-set$SCORESET file to be ignored and not written to the
>> 72_scores.cf.
>> 
>> The dos version of the script:
>>
>http://svn.apache.org/repos/asf/spamassassin/trunk/rulesrc/sandbox/dos/new-rule-score-gen/do-nightly-rescore-example
>> Also has the $SCORESET param, but it does not seem to be set anywhere
>so
>> it might have functioned as if no param was given.
>> 
>> To me it sounds illogical to ignore the net rules, they are just
>> masschecked less often, but should still be taken into consideration,
>> right?
>> 
>> 
>> And I just noticed: scores-set1 is still truncated at MILLION_USD,
>but
>> maybe  because the masscheck is only once a week?
>> 
>> 
>
>Not sure either.  I would like to get something to work even if it's
>not 
>perfect so we can get sa-update updating again with "good enough" 
>72_scores.cf which would be better than nothing.  We can't roll out any
>
>new rules at all right now.
>
>Then I would love to carefully redesign the scripts so they could be
>run 
>hourly instead of daily and only run if there is an actual
>change/commit 
>waiting to be tested.  The new script would be more modular and share 
>functions/logic instead of each one doing nearly identical things but 
>slightly differently.  The scripts would have proper logging with 
>meaningful return codes and not use "set -x" and "|| exit $?" which is 
>extremely difficult to troubleshoot.
>
>Right now with the current problem of commits messing up the masscheck 
>processing, it's taking days and weeks to troubleshoot problems as we 
>make changes to try to improve things.  This makes the project seem
>like 
>it's not doing anything when we are really trying hard to improve
>things.
>
>We need to add new features and improve SA or rspamd is going to take 
>over and leave SA behind.  I have some ideas and plans that I would
>love 
>to implement over time to help the entire SA install base but we have
>to 
>have a solid foundation first.
>
>
 Merijn,

 I am confused as to what is trying to be accomplished with the
>patch on
 but 6400.  Maybe I am misunderstanding something.  It looks like
>these
 MSPIKE scores need to remain in 50_scores.cf since they are tied to
>a
 specific SA version check > 3.4.0.
>>>
>>> Normally, all rules which are in 50_scores.cf will never get
>rescored
>>> scores in 72_scores.cf.
>>>
>>> The script to which the patch applies does this, but due to a too
>strict
>>> regex, it doesn't see any score line if its prepended with
>whitespace.
>>>

 Do we need to remove them from 72_scores.cf, note that in the bug
>then
 close it?
>>>
>>> Once the patch is applied, the mailspike rules should vanish from
>>> 72_scores.cf by itself.
>>>

 Dave
>>>
>>>
>> 
>> 

Re: mailspike in 72_scores

[Dave Jones]

On 11/05/2017 10:26 AM, Merijn van den Kroonenberg wrote:

Note to self: investigate why in do-nightly-rescore-example.sh is called
like this:
merge-scoresets $SCORESET
With a $SCORESET param. It will cause all rules which are not in that
scores-set$SCORESET file to be ignored and not written to the
72_scores.cf.

The dos version of the script:
http://svn.apache.org/repos/asf/spamassassin/trunk/rulesrc/sandbox/dos/new-rule-score-gen/do-nightly-rescore-example
Also has the $SCORESET param, but it does not seem to be set anywhere so
it might have functioned as if no param was given.

To me it sounds illogical to ignore the net rules, they are just
masschecked less often, but should still be taken into consideration,
right?


And I just noticed: scores-set1 is still truncated at MILLION_USD, but
maybe  because the masscheck is only once a week?




Not sure either.  I would like to get something to work even if it's not 
perfect so we can get sa-update updating again with "good enough" 
72_scores.cf which would be better than nothing.  We can't roll out any 
new rules at all right now.


Then I would love to carefully redesign the scripts so they could be run 
hourly instead of daily and only run if there is an actual change/commit 
waiting to be tested.  The new script would be more modular and share 
functions/logic instead of each one doing nearly identical things but 
slightly differently.  The scripts would have proper logging with 
meaningful return codes and not use "set -x" and "|| exit $?" which is 
extremely difficult to troubleshoot.


Right now with the current problem of commits messing up the masscheck 
processing, it's taking days and weeks to troubleshoot problems as we 
make changes to try to improve things.  This makes the project seem like 
it's not doing anything when we are really trying hard to improve things.


We need to add new features and improve SA or rspamd is going to take 
over and leave SA behind.  I have some ideas and plans that I would love 
to implement over time to help the entire SA install base but we have to 
have a solid foundation first.




Merijn,

I am confused as to what is trying to be accomplished with the patch on
but 6400.  Maybe I am misunderstanding something.  It looks like these
MSPIKE scores need to remain in 50_scores.cf since they are tied to a
specific SA version check > 3.4.0.


Normally, all rules which are in 50_scores.cf will never get rescored
scores in 72_scores.cf.

The script to which the patch applies does this, but due to a too strict
regex, it doesn't see any score line if its prepended with whitespace.



Do we need to remove them from 72_scores.cf, note that in the bug then
close it?


Once the patch is applied, the mailspike rules should vanish from
72_scores.cf by itself.



Dave

Re: mailspike in 72_scores

[Merijn van den Kroonenberg]

Note to self: investigate why in do-nightly-rescore-example.sh is called
like this:
merge-scoresets $SCORESET
With a $SCORESET param. It will cause all rules which are not in that
scores-set$SCORESET file to be ignored and not written to the
72_scores.cf.

The dos version of the script:
http://svn.apache.org/repos/asf/spamassassin/trunk/rulesrc/sandbox/dos/new-rule-score-gen/do-nightly-rescore-example
Also has the $SCORESET param, but it does not seem to be set anywhere so
it might have functioned as if no param was given.

To me it sounds illogical to ignore the net rules, they are just
masschecked less often, but should still be taken into consideration,
right?


And I just noticed: scores-set1 is still truncated at MILLION_USD, but
maybe  because the masscheck is only once a week?


>> Merijn,
>>
>> I am confused as to what is trying to be accomplished with the patch on
>> but 6400.  Maybe I am misunderstanding something.  It looks like these
>> MSPIKE scores need to remain in 50_scores.cf since they are tied to a
>> specific SA version check > 3.4.0.
>
> Normally, all rules which are in 50_scores.cf will never get rescored
> scores in 72_scores.cf.
>
> The script to which the patch applies does this, but due to a too strict
> regex, it doesn't see any score line if its prepended with whitespace.
>
>>
>> Do we need to remove them from 72_scores.cf, note that in the bug then
>> close it?
>
> Once the patch is applied, the mailspike rules should vanish from
> 72_scores.cf by itself.
>
>>
>> Dave
>
>

Cron <automc@sa-vm1> ~/svn/trunk/build/mkupdates/run_nightly | /usr/bin/tee /var/www/automc.spamassassin.org/mkupdates/mkupdates.txt

[Cron Daemon]

+ promote_active_rules
+ pwd
+ /usr/bin/perl build/mkupdates/listpromotable
/usr/local/spamassassin/automc/svn/trunk
HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
day 1 contains a --net mass-check! offsetting by an extra day
Use of "goto" to jump into a construct is deprecated at 
build/mkupdates/listpromotable line 85.
HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/3-days-ago?xml=1
no 'mcviewing', 'mcsubmitters' microformats on day 2
URL: http://ruleqa.spamassassin.org/3-days-ago?xml=1
+ exit 25