as background, forwarding this thread, since Doc indicated towards the end
that he meant it to go to dev anyway ;)
--j.
------- Forwarded Messages
Date: Tue, 28 Feb 2006 18:26:08 -0600
From: Doc Schneider <[EMAIL PROTECTED]>
To: Justin Mason <[EMAIL PROTECTED]>
cc: Duncan Findlay <[EMAIL PROTECTED]>,
Theo Van Dinter <[EMAIL PROTECTED]>
Subject: Re: Issues with nightly runs & rsync, docs, etc.
Justin Mason wrote:
> Duncan Findlay writes:
>> On Sun, Feb 26, 2006 at 11:47:34PM -0500, Theo Van Dinter wrote:
>>> On Sun, Feb 26, 2006 at 11:32:57PM -0500, Duncan Findlay wrote:
>>>>> So the problem here is that mkrules is never run, so the majority
>>>>> of rules aren't actually run. I've added in a mkrules call into the
>>>>> nightlymc script.
>>>> Umm... which nightlymc script is this? Also, we don't all use the same
>>>> script for running nightlies, so if you change something make sure you
>>>> let people know very clearly. :-)
>>> The one that runs on the zones machine to generate the rsync image.
>>> http://wiki.apache.org/spamassassin/NightlyMassCheck simply says that
>>> people can rsync down the appropriate version and start running, which
>>> wasn't really true since you had to run mkrules as well, so now it's
>>> run at the server so when people rsync they get the full image.
>> Ah... OK...
>>
>> I'd forgotten we even allow rsync download of the tree for this
>> purpose, so I had no idea what you were talking about :-)
>
> I'm thinking maybe we should just all standardize on one method;
> probably rsync-based. All these different versions are not helping.
>
> --j.
I have a question on doing nightly mass-checks.
Have rsync and SVN access so these are a non-issue.
Anyway, how "current" is the wiki page on doing these mass-checks? And I
presume doing them on a SVN release would be the best?
Should I build an svn version grabbing all of it to a new place to build
it, of course. And then point it to my ham and spam corpus (I know there
are switches in mass-check to point to them)?
So in other words build an SVN version for a single user system and make
& make install it? What about the makerules?
I'm more than willing to fix the wiki page if I better understood the
process.
Thanks!
--
-Doc
SA/SARE/URIBL/SURBL -- Ninja
6:12pm up 38 days, 15:32, 15 users, load average: 0.19, 0.30, 0.35
SARE HQ http://www.rulesemporium.com/
------- Message 2
Date: Tue, 28 Feb 2006 20:27:01 -0500
From: Theo Van Dinter <[EMAIL PROTECTED]>
To: Doc Schneider <[EMAIL PROTECTED]>
cc: Justin Mason <[EMAIL PROTECTED]>, Duncan Findlay <[EMAIL PROTECTED]>
Subject: Re: Issues with nightly runs & rsync, docs, etc.
--8OLezhwaZ8w3s/dZ
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Feb 28, 2006 at 06:26:08PM -0600, Doc Schneider wrote:
> I have a question on doing nightly mass-checks.
> Have rsync and SVN access so these are a non-issue.
Sure.
> Anyway, how "current" is the wiki page on doing these mass-checks? And I
> presume doing them on a SVN release would be the best?
I don't know, but I've been thinking that they need to be updated.
> Should I build an svn version grabbing all of it to a new place to build
> it, of course. And then point it to my ham and spam corpus (I know there
> are switches in mass-check to point to them)?
You don't need to do a build, just grab the appropriate version. If you use
the rsync method, you should be all set. If you use svn, you'll need to do
something like "rm -f rules/* ; svn -r <version> up ; build/mkrules --out
rules".
There aren't really options to point at them, but there are parameters of
course.
> I'm more than willing to fix the wiki page if I better understood the
> process.
That'd be great, thanks! I've been meaning to do something with it but
haven't gotten around to it yet.
Basically the process goes:
- On your own time or whenever, prep your corpus into ham and spam. You can
have mbx and mbox files (multiple messages in one file), file (single
message in file), and dir (a directory of "file" type files). This is
mostly covered in http://wiki.apache.org/spamassassin/MassCheck
- Currently, Sunday through Friday are "nightly" runs. Saturday is the
"weekly" run and requires the addition of "--net --reuse" to mass-check.
- After 0900 GMT (iirc), you should update your SA corpus area to the
appropriate version. This is either: rsync the appropriate area
(rsync.spamassassin.org::tagged_builds/{nightly,weekly}_mass_check). Or,
grab the http://rsync.spamassassin.org/{nightly,weekly}-versions.txt file,
and grab the version out of the last line in the file. For 3.2, "rm -f
rules/*", then do the "svn up -r <version>", then run "build/mkrules --out
rules".
- run mass-check. information is all over the place, but something like:
masses/mass-check --progress --all -c rules -n <messages>
where <messages> is something like "ham:mbox:/path/to/mbox
spam:dir:/path/to/spam/dir".
- when it's done, rsync the resulting ham.log and spam.log files to
rsync.spamassassin.org::corpus/{ham,spam}-<YOURUSERNAME>.log
I think most of this stuff is documented, though not completely. This
probably confuses new folks. :(
BTW, I put in my latest scripts to do the nightly/weekly runs in the
contrib/ area in 3.2 which may shed some more light on things. In the
end, I like jm's idea of standardizing on rsync. It makes that part of
things easier.
--
Randomly Generated Tagline:
"If you want to waste food, throw a vegetable." - The Drew Carey Show
--8OLezhwaZ8w3s/dZ
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (GNU/Linux)
iD8DBQFEBPhkvTLro+k8grsRAvoSAJ4nKVmooIx19Z+CAZt/AeCZvUh98wCeM/m0
vz64DpQNgljzOF6W3Bkw9Z4=
=qV6B
-----END PGP SIGNATURE-----
--8OLezhwaZ8w3s/dZ--
------- Message 3
Date: Tue, 28 Feb 2006 20:22:10 -0600
From: Doc Schneider <[EMAIL PROTECTED]>
To: Theo Van Dinter <[EMAIL PROTECTED]>
cc: Justin Mason <[EMAIL PROTECTED]>, Duncan Findlay <[EMAIL PROTECTED]>
Subject: Re: Issues with nightly runs & rsync, docs, etc.
Theo Van Dinter wrote:
> On Tue, Feb 28, 2006 at 06:26:08PM -0600, Doc Schneider wrote:
>> Should I build an svn version grabbing all of it to a new place to build
>> it, of course. And then point it to my ham and spam corpus (I know there
>> are switches in mass-check to point to them)?
>
> You don't need to do a build, just grab the appropriate version. If you use
> the rsync method, you should be all set. If you use svn, you'll need to do
> something like "rm -f rules/* ; svn -r <version> up ; build/mkrules --out
> rules".
Something I'm not clear on here, there are nightly builds available via
rsync for doing mass-checks? And those are all set to rock and roll?
> There aren't really options to point at them, but there are parameters of
> course.
>
>> I'm more than willing to fix the wiki page if I better understood the
>> process.
>
> That'd be great, thanks! I've been meaning to do something with it but
> haven't gotten around to it yet.
>
> Basically the process goes:
I'll do a few mass-checks and see what is all needed than work on fixing
the wiki.
> I think most of this stuff is documented, though not completely. This
> probably confuses new folks. :(
>
> BTW, I put in my latest scripts to do the nightly/weekly runs in the
> contrib/ area in 3.2 which may shed some more light on things. In the
> end, I like jm's idea of standardizing on rsync. It makes that part of
> things easier.
>
I did an svn up and got your new files under /contrib .
Looking through those files I think maybe a short README is needed to
tell what each of those do. (ie. which order should those be run in to
do a mass-check)
8*))
--
-Doc
SA/SARE -- Ninja
8:12pm up 38 days, 17:32, 15 users, load average: 0.18, 0.49, 0.75
SARE HQ http://www.rulesemporium.com/
------- Message 4
Date: Tue, 28 Feb 2006 21:48:05 -0500
From: Theo Van Dinter <[EMAIL PROTECTED]>
To: Doc Schneider <[EMAIL PROTECTED]>
cc: Justin Mason <[EMAIL PROTECTED]>, Duncan Findlay <[EMAIL PROTECTED]>
Subject: Re: Issues with nightly runs & rsync, docs, etc.
--NHTWBGw6kTP4k5lb
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Tue, Feb 28, 2006 at 08:22:10PM -0600, Doc Schneider wrote:
> Something I'm not clear on here, there are nightly builds available via
> rsync for doing mass-checks? And those are all set to rock and roll?
Yes. It's mentioned on http://wiki.apache.org/spamassassin/NightlyMassCheck
> I'll do a few mass-checks and see what is all needed than work on fixing
> the wiki.
:) Please let me/us know if there are any questions. BTW, posting to dev@
may be a better for this type of discussion.
I do believe the wiki and the build/README* files have most of the informat
ion
necessary to do the mass-checks. The only new thing is the mkrules bit if
you're doing SVN updates.
> I did an svn up and got your new files under /contrib .
> Looking through those files I think maybe a short README is needed to
> tell what each of those do. (ie. which order should those be run in to
> do a mass-check)
Sure, I can see about writing something up. Mine isn't the end-all set
of scripts, and they're probably a bit overtuned for my environment and
a little more complex than is necessary for folks just starting out.
I know there's another one or two script sets in the tree as well from
various folks, btw.
--
Randomly Generated Tagline:
"... one of the main causes of the fall of the Roman Empire was that,
lacking zero, they had no way to indicate successful termination of their
C programs." - Robert Firth
--NHTWBGw6kTP4k5lb
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.1 (GNU/Linux)
iD8DBQFEBQtlvTLro+k8grsRAiLNAJ99bLV+ThXpih8RB3APyDmifPld9wCeNq4w
rAC1BFNSQLsWWV+zFV1wsps=
=oevT
-----END PGP SIGNATURE-----
--NHTWBGw6kTP4k5lb--
------- Message 5
Date: Tue, 28 Feb 2006 21:09:31 -0600
From: Doc Schneider <[EMAIL PROTECTED]>
To: Theo Van Dinter <[EMAIL PROTECTED]>
cc: Justin Mason <[EMAIL PROTECTED]>, Duncan Findlay <[EMAIL PROTECTED]>
Subject: Re: Issues with nightly runs & rsync, docs, etc.
Theo Van Dinter wrote:
> On Tue, Feb 28, 2006 at 08:22:10PM -0600, Doc Schneider wrote:
>> Something I'm not clear on here, there are nightly builds available via
>> rsync for doing mass-checks? And those are all set to rock and roll?
>
> Yes. It's mentioned on http://wiki.apache.org/spamassassin/NightlyMassCheck
I'll have to wander over there and check that out.
>> I'll do a few mass-checks and see what is all needed than work on fixing
>> the wiki.
>
> :) Please let me/us know if there are any questions. BTW, posting to dev@
> may be a better for this type of discussion.
ACK! I thought I had the dev@ list in the CC: but guess not. I'll sure
move some of this over to there.
> I do believe the wiki and the build/README* files have most of the informatio
n
> necessary to do the mass-checks. The only new thing is the mkrules bit if
> you're doing SVN updates.
AHA! Thanks, I always try to read all the README's but missed that one.
>> I did an svn up and got your new files under /contrib .
>> Looking through those files I think maybe a short README is needed to
>> tell what each of those do. (ie. which order should those be run in to
>> do a mass-check)
>
> Sure, I can see about writing something up. Mine isn't the end-all set
> of scripts, and they're probably a bit overtuned for my environment and
> a little more complex than is necessary for folks just starting out.
> I know there's another one or two script sets in the tree as well from
> various folks, btw.
>
Thanks, I think even a small README in there would help. Even if yours
isn't the "end-all" one never hurts to Document! Of course some of us do
need to learn to find -name 'README*' HAR!
Thanks,
--
-Doc
SA/SARE -- Ninja
9:04pm up 38 days, 18:24, 15 users, load average: 0.66, 0.69, 0.58
SARE HQ http://www.rulesemporium.com/
------- End of Forwarded Messages
