Daryl C. W. O'Shea writes: > Justin Mason wrote: > > Daryl C. W. O'Shea writes: > >> Justin Mason wrote: > > >> At a minimum I'd disable the 3.2 updates until we figure out exactly > >> > what we want to do (actually that's what I intended). > > > > OK, sounds fine. > > So, to do so, I'd just create an update that contains an identical > ruleset to the one being released with 3.2.0, using say 533971 as the > update number and then add the appropriate txt record to the zone file: > > 0.2.3.updates.spamassassin.org. 3600 IN TXT "533971"
Actually, the cron jobs run as "updatesd" on the zone would quickly move on and create another update tomorrow, so that wouldn't help. ;) I've edited the 'build/mkupdates/run_part2' script, which is run by cron, and set it to write the updates to a different dir ( http://buildbot.spamassassin.org/updatedev/ ) and to a different record in the DNS zone ( 0.2.3.updates.dev.spamassassin.org ). The existing files/records will remain the same until we edit the script again and uncomment the "live" lines; in the meantime, we'll be able to see what *would* be going into the updates by checking those locations. > > On top of that, there were lint failures caused (iirc) by plugin > > dependencies being missed; and false positives caused by too-high scores, > > when multiple rules had good 'freqs' but were matching the same message > > subset. (mkrules already takes care of the former, and running a rescorer > > like the GA would have fixed the latter.) > > FWIW, the rescoring I've got running now seems to be producing sane > scores. I've been using the generated set1 scores for set3 with no > complaints so far. Good news. (it should be measurable anyway, since we can run set3 mass-checks with those scores against our corpora.) > >> In any case I'd rather have automatic updates too, I'm just against > >> releasing the current auto updates the way they are. I really only > >> meant to switch the 3.2 updates to 3.3 to stop the current auto-updates > >> for the stable release (for now) and leave a 3.2 update place holder in > >> DNS with no real updates (until we get acceptable auto-updates). > >> > >> > >>> I'd prefer to have the auto-promotion, updates generation, etc. to > >>> output rules directly to the 3.2.x updates. > >> The auto-promotion has some issues: > >> > >> - auto-demotion needs some dampening (say a three day minimum before > >> being removed or at least a lower criteria for keeping currently > >> promoted rules); rules are bouncing in and out of the active.list way > >> too much; of particular interest are rules bouncing in and out around > >> the weekly non-net, net, non-net 3 night sequence > > > > Yeah, good point. > > > >> - auto-promoted rules need auto-generated scores; there's been a number > >> of instances of duplicate rules with 'big' scores causing FPs in the > >> manual updates... and somebody had to manually add those rules and > >> scores without noticing the problem, doing the same thing > >> automatically can't be any better > > > > +1. No, I agree -- the auto-generated scores are essential to avoid > > the FP situation I mentioned above. > > I'm happy with using the generated scores I've got running now. I'm > open to comments, changes, or something different though. I haven't looked into them in massive detail yet, but as far as I can see it looks good... > >> - at some point trunk, which the updates are currently based on, is > >> probably going to diverge in compatibility from 3.2; that's not an > >> issue now but down the road there's potential for breakage... any > >> automated process should try to at least lint the updates with all > >> targeted versions of SA (this still applies if the updates become > >> based on stable branch mass checks and not trunk) > > > > I'm not too worried about that... trunk can break ;) > > I'm not concerned about breaking trunk, rather, I'm concerned about > using trunk to generate the stable branch updates and breaking the > stable updates. I can't think of anything obvious that the client side > sa-update lint wouldn't catch, but even having the client update detect > the update as being bad is, well, bad. > > Regardless of what the updates are based on, really, there should be a > step in the automated process that attempts to lint the update against > every version that the update is targeted for. In the event of a > failure the update won't be published (maybe it already does this, but > I'd don't remember it doing so). I was actually thinking of this model: - 3.1.x updates: remain as they are, manually updated. - the current nightly auto-update generation switches to generating updates for 3.2.x; and our nightly mass-checks are m-c'd using the code in the 3.2.x maintainance branch; in other words, *rule* development runs against 3.2.x, not against trunk. - trunk has no nightly mass-check. (at least initially) After all, there's no real reason to run nightly mass-checks against trunk; we want nightly mass-checks to QA rules, and the rules should be targeted against 3.2.x. (We could always set up one or two new mass-check crons to m-c trunk, but that can wait.) > In any case, this one isn't a show stopper, it's something that I can > look at implementing post 3.2.0 release. OK, cool. agreed. --j.
