checkSAupdateMirrors.sh on sa-vm1.apache.org - 1 mirror DOWN, 0 mirrors STALE

[root]

Fetching sa-update URLs from http://spamassassin.apache.org/updates/MIRRORED.BY

http://sa-update.dnswl.org/: UP (CURRENT)

http://www.sa-update.pccc.com/: UP (CURRENT)

http://sa-update.secnap.net/: UP (CURRENT)

http://sa-update.space-pro.be/: UP (CURRENT)

http://sa-update.ena.com/: DOWN

updateDNS.sh on sa-vm1.apache.org - DNS updates disabled

[noreply]

3.3.3.updates (TXT) -> \"1814708\"

File /usr/local/bin/updateDNS.disabled exists, not updating DNS.

Re: spamassassin.org

[Kevin A. McGrail]

On 11/9/2017 12:06 PM, Greg Stein wrote:


The short answer is: the PMC is in charge. Not a problem. My concern 
is the reliance on an informal arrangement with 3rd parties that 
underlies the SA operations. I've shared that concern, and am now 
satisfied :-)


Perfectly valid concern that I share as well.  I am considering the DNS 
settled from my perspective but with let you and the PMC and Chair 
decide for sure.


More data:

#1 Sonic is slated to be recognized as a bronze directed sponsor* and we 
have our first sponsor (silver) signing using the new sponsor agreement 
I drafted.  I've been talking to Sonic for a few months now and making 
it not informal for the same concern.


#2 ENA & PCCC likely should get on that bandwagon as well though I will 
recuse myself on the PCCC discussion.  And I'm not concerned about my 
firm so I'll get that ducked in a row.


#3 Namecheap's relatively high TTL's and lack of support for things like 
rbldnsd don't make it a suitable candidate to be involved in the DNS for 
the SA project in any foreseeable future. I'd be comfortable saying 2+ 
years.


Regards,

KAM


*Draft of the new sponsor thanks page.  It's a WIP: 
http://www.staging.apache.org/foundation/thanks2.html

Re: spamassassin.org

[Greg Stein]

On Thu, Nov 9, 2017 at 10:44 AM, Kevin A. McGrail  wrote:
>...

> Re: moving the NS records, I'm likely one of the few PMC members *[Working
> to change that with our sysadmins group] left that can speak to this issue
> but defer to a vote.
>
> TL;DR: Don't be in such a hurry to put SA DNS onto ASF Infra. It might
> cause a lot of grief and the grief is currently handled so it has zero net
> gain for a lot of work.
>
I'm in no hurry, and not forcing any change. It's on you guys, what you'd
like to do. I merely asked for consideration. "Status quo" is totally fine.

> Overall, I don't support changing the status quo and here are the reasons:
>
> - We have just rearchitected around PowerDNS for API calls.  Switching
> would be difficult but not impossible.  But I imagine we won't have APIs
> under Infra.  Ignore that issue for now while you read more bullet points
> below.
>
Namecheap has an API.

(I've been using it for domain management; found it easy to use, and
customer/API support has been excellent; I believe that over time, we'll
also use it to manage apache.org itself)

> - I don't think it's clear that the master DNS for spamassassin.org is on
> ASF infra as a hidden master now.  It has ALWAYS been there since the
> project moved under the ASF.
>
> - The name servers today share the load and use distributed DNS to Sonic,
> PCCC & ENA.  How is consolidating a distributed, resilient DNS system going
> to improve things?  I'd argue you are putting all the eggs in one basket
> and it's less viable.
>
Oh, haha... no way am I suggesting we move this to ASF provision. My
current plan is to try and shift all ASF DNS provision over to Namecheap,
who provides an SLA of 100% [1]. I am happy with that one basket, and we'll
be heading in that direction, to reduce our own management/costs for DNS.

>...

The short answer is: the PMC is in charge. Not a problem. My concern is the
reliance on an informal arrangement with 3rd parties that underlies the SA
operations. I've shared that concern, and am now satisfied :-)

Cheers,
-g

[1] https://www.namecheap.com/security/premiumdns.aspx

Re: Eureka: truncation of 72_active.cf

[Merijn van den Kroonenberg]

> On 11/09/2017 07:27 AM, Merijn van den Kroonenberg wrote:
>>
> Well...  My local patch is still in place.  This is lines 195-206 of
> the
> current trunk/masses/rule-update-score-gen/generate-new-scores.sh:

 What is the full path? The question is in which working copy did you
 apply
 this change (what is it used for). Because in the working directories
 used
 for score generation it isn't. Thats in
 /usr/local/spamassassin/automc/tmp/generate-new-scores/trunk-new-rules-set0
 (each set having its fully checked out working copy)
 In generate-new-scores.sh the workingcopies are done each night
 completely
 from scratch (old one removed, new one checked out). So no uncommitted
 code can be used in the score generation part.

>>>
>>> /usr/local/spamassassin/automc/svn/trunk/build/mkupdates/do-stable-update-with-scores
>>>
>>> should be calling
>>>
>>> /usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/do-nightly-rescore-example.sh
>>>
>>> which finally calls the locally modified and not committed script that
>>> is intended to create our 72_active_before_grep.cf but it's not some
>>> something is definitely not as I expected
>>>
>>> /usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/generate-new-scores.sh
>>
>> Very strange, no idea why it doesn't work and nothing gets logged in the
>> mail.
>>
>> can you do a:
>> svn status /usr/local/spamassassin/automc/svn/masses
>> and
>> svn status /usr/local/spamassassin/automc/svn/trunk
>>
>
> Both commands return nothing.  I need to investigate this closer for
> sure.

Thats weird, when it returns nothing it means there are no local changes
in the checkout.
Something which could affect this is the svn:ignore property, which
contains quite a list, but I don't really spot something which could
affect our case.

Unless ofcource, your changes are really gone now?

> Day job and family stuff is currently keeping me busy.  It's not
> easy to tshoot this when it takes a long time between runs.  Can't stay
> focused on it.  I could run these scripts manually in the evenings US
> time but it still takes about 40 minutes if I remember correctly.  It
> did this a lot back in April and May when I could stay on top of it to
> make some progress fairly quickly.

If I would enroll into apache, would I get ssh access so I could look
around myself?

>
> Another major problem we have to figure out is the rules promotion
> script from the build/mkupdates/run_nightly.  If we can't get any rules
> promoted, then masscheck'ing is just spinning it's wheels since we need
> and updates active.list to tell what the current 72_scores.cf should
> contain.

Tomorrow I have a day of (they are installing something at home, which
should not require my full attention). So I can dive into this tomorrow.

>
> + promote_active_rules
> + pwd
> /usr/local/spamassassin/automc/svn/trunk
> + /usr/bin/perl build/mkupdates/listpromotable
> HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
> HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
> no 'mcviewing', 'mcsubmitters' microformats on day 2
> URL: http://ruleqa.spamassassin.org/2-days-ago?xml=1
> + exit 25
>
>>>
>>> All of those script should be checking out fresh copies of everything
>>> they work with into
>>>
>>> /usr/local/spamassassin/automc/tmp
>>>
>>>
>>
>>
>
>

Re: Eureka: truncation of 72_active.cf

[Dave Jones]

On 11/09/2017 07:27 AM, Merijn van den Kroonenberg wrote:



Well...  My local patch is still in place.  This is lines 195-206 of
the
current trunk/masses/rule-update-score-gen/generate-new-scores.sh:


What is the full path? The question is in which working copy did you
apply
this change (what is it used for). Because in the working directories
used
for score generation it isn't. Thats in
/usr/local/spamassassin/automc/tmp/generate-new-scores/trunk-new-rules-set0
(each set having its fully checked out working copy)
In generate-new-scores.sh the workingcopies are done each night
completely
from scratch (old one removed, new one checked out). So no uncommitted
code can be used in the score generation part.



/usr/local/spamassassin/automc/svn/trunk/build/mkupdates/do-stable-update-with-scores

should be calling

/usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/do-nightly-rescore-example.sh

which finally calls the locally modified and not committed script that
is intended to create our 72_active_before_grep.cf but it's not some
something is definitely not as I expected

/usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/generate-new-scores.sh


Very strange, no idea why it doesn't work and nothing gets logged in the
mail.

can you do a:
svn status /usr/local/spamassassin/automc/svn/masses
and
svn status /usr/local/spamassassin/automc/svn/trunk



Both commands return nothing.  I need to investigate this closer for 
sure.  Day job and family stuff is currently keeping me busy.  It's not 
easy to tshoot this when it takes a long time between runs.  Can't stay 
focused on it.  I could run these scripts manually in the evenings US 
time but it still takes about 40 minutes if I remember correctly.  It 
did this a lot back in April and May when I could stay on top of it to 
make some progress fairly quickly.


Another major problem we have to figure out is the rules promotion 
script from the build/mkupdates/run_nightly.  If we can't get any rules 
promoted, then masscheck'ing is just spinning it's wheels since we need 
and updates active.list to tell what the current 72_scores.cf should 
contain.


+ promote_active_rules
+ pwd
/usr/local/spamassassin/automc/svn/trunk
+ /usr/bin/perl build/mkupdates/listpromotable
HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
no 'mcviewing', 'mcsubmitters' microformats on day 2
URL: http://ruleqa.spamassassin.org/2-days-ago?xml=1
+ exit 25



All of those script should be checking out fresh copies of everything
they work with into

/usr/local/spamassassin/automc/tmp

Re: Eureka: truncation of 72_active.cf

[Merijn van den Kroonenberg]

>>> Well...  My local patch is still in place.  This is lines 195-206 of
>>> the
>>> current trunk/masses/rule-update-score-gen/generate-new-scores.sh:
>>
>> What is the full path? The question is in which working copy did you
>> apply
>> this change (what is it used for). Because in the working directories
>> used
>> for score generation it isn't. Thats in
>> /usr/local/spamassassin/automc/tmp/generate-new-scores/trunk-new-rules-set0
>> (each set having its fully checked out working copy)
>> In generate-new-scores.sh the workingcopies are done each night
>> completely
>> from scratch (old one removed, new one checked out). So no uncommitted
>> code can be used in the score generation part.
>>
>
> /usr/local/spamassassin/automc/svn/trunk/build/mkupdates/do-stable-update-with-scores
>
> should be calling
>
> /usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/do-nightly-rescore-example.sh
>
> which finally calls the locally modified and not committed script that
> is intended to create our 72_active_before_grep.cf but it's not some
> something is definitely not as I expected
>
> /usr/local/spamassassin/automc/svn/masses/rule-update-score-gen/generate-new-scores.sh

Very strange, no idea why it doesn't work and nothing gets logged in the
mail.

can you do a:
svn status /usr/local/spamassassin/automc/svn/masses
and
svn status /usr/local/spamassassin/automc/svn/trunk

>
> All of those script should be checking out fresh copies of everything
> they work with into
>
> /usr/local/spamassassin/automc/tmp
>
>

Cron <automc@sa-vm1> ~/svn/trunk/build/mkupdates/run_nightly | /usr/bin/tee /var/www/automc.spamassassin.org/mkupdates/mkupdates.txt

[Cron Daemon]

+ promote_active_rules
+ pwd
/usr/local/spamassassin/automc/svn/trunk
+ /usr/bin/perl build/mkupdates/listpromotable
HTTP get: http://ruleqa.spamassassin.org/1-days-ago?xml=1
HTTP get: http://ruleqa.spamassassin.org/2-days-ago?xml=1
no 'mcviewing', 'mcsubmitters' microformats on day 2
URL: http://ruleqa.spamassassin.org/2-days-ago?xml=1
+ exit 25