-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Duncan Findlay writes: > On Wed, Jul 20, 2005 at 03:13:29PM -0700, Loren Wilton wrote: > > This is a very interesting idea that I think needs more exploring in the > > future. Any SA server that has a Bayes database potentially has most of the > > knowledge to be able to participate in Seti-like background processing for > > determining rule hit ratios. For that matter, any SA server should be able > > to collect logs of the rules that are hitting there, and send out that rule > > hit information to some central server once a day. This won't necessarily > > give fp/fn hit counts, but it can give total hits per rule, and that is > > moderately valuable information in itself, while still being pretty > > annonomous. > > Interesting, I agree. I'm not sure this will help at all with new rule > development, but it would give us interesting data over relative hit > rates over time. It would certainly be lots of work to set up, though. :-( > > > Sare has a rule scoring method that Bob developed that assigns a probable > > score to the rule based on the masscheck results. Sometimes we modify this > > manually based on other factors, but most of the time it goes into the rules > > files directly. We know it isn't as good as a full SA scoring run. But on > > the other hand, it doesn't require a full SA scoring run, and generally > > produces pretty usable results. I would envision smething like this being > > used for initial rule introductions, and periodically the rules would be > > rescored using a full scoring run. > > Even better would be to be able to do a full scoring run every night, > or every week or something like that, but this would be very difficult > to achieve. Perhaps we can look at the results after the 3.1 run and > see if there are any relationships we can use between rules hit rates > and score. I fear that there's too much interdependency though for > this to be possible. BTW we should really get Henry to comment on this stuff, he's the expert! ;) - --j. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.5 (GNU/Linux) Comment: Exmh CVS iD8DBQFC39QbMJF5cimLx9ARAhR/AJ9UHHBupMH2jb+SelMg41NiUtsFAACeNr82 9zkmE35YJIG8ANd6Cq3UClg= =akfU -----END PGP SIGNATURE-----
