Theo Van Dinter writes:
> On Wed, Oct 18, 2006 at 09:06:15AM -0700, Ken A wrote:
> > Any thoughts on how to best address this issue, other than every SA
> > admin on the planet writing their own rules every evening? I'd note that
> > these are not identical spams, but are somewhat limited in their
> > 'variety', and they are hitting DCC now.
>
> Welcome to anti-spam rule development. :) These are no different than
> anything before... People need to be paying attention to what's being
> sent/received. Rules need to be written and tested. The good ones can
> be distributed.
>
> Via sa-update, new rules can be distributed very quickly, so it's really
> about the time necessary to develop and test new rules, which generally
> speaking comes down to manpower (there's some technology involved as well,
> but that's addressable).
Actually, on this point, I had an idea.
Currently, we have this worst-case scenario with sa-update:
- day 1: 0930 UTC: developer writes good rule, checks it into
rulesrc/sandbox/dev/20_whatever.cf
- day 1: [nothing interesting happens until next nightly-mass-check tag]
- day 2: 0900 UTC: SVN is tagged; all mass-checkers check out of SVN and
start mass-checks
- day 2: [allow time for mass-checks]
- day 3: 0830 UTC: updatesd runs "build/mkupdates/run_nightly", collates
mass-check results, adds rule to "rules/active.list"
- day 3: 0850 UTC: new update is published, containing the rule
So, in other words, the worst-case scenario is that it'd take just under 2
days to get the rule into a packaged, released update.
I was thinking we could have a fast-reaction mode for just-created rules:
- day 1: 0930 UTC: developer writes good rule, checks it into
rulesrc/sandbox/dev/20_whatever.cf
- day 1: 0931 UTC: bbmass preflight mass-check runs; new rule gets 1.0
S/O on that limited corpus set
- day 2: 0830 UTC: updatesd runs "build/mkupdates/run_nightly", collates
mass-check results, adds rule to "rules/active.list"; also, adds
newly-added rules that scored 1.0 S/O on the most recent preflight
mass-check, and are not appearing in the nightly mass-check results
yet
- day 2: 0850 UTC: new update is published, containing the rule
in other words, reducing the worst-case scenario to just under 1 day. (If
we were to increase frequency of update publishing in the future, that
would then reduce that further, if necessary.)
Rules that got promoted based on "being new" and having a 1.0 S/O in the
preflight mass-checks would then only *stay* promoted if they then passed
the normal, existing promotion criteria -- so a rule that was good
"enough" to get into the update due to a 1.0 S/O, but had FPs on the
larger test set, would fall out anyway after 1 day.
--j.
