hey John -- that URI issue is now fixed.
regarding overlap: the log file on spamassassin2 , /home/automc/freqsd/log , has this for the most recent OVERLAP file generation: creating: /export/home/ruleqadb/20100321/r925742-n/OVERLAP.new (OVERLAP) started Wed Mar 31 12:26:07 2010... so it appears to be backlogged by 10 days. :( It appears we don't have the CPU time to keep up with generating these on spamassassin2 this way. I've stopped and restarted the freqsd anyway, in an effort to persuade it to concentrate on more recent logs. (I have no idea if this will work.) To fix the issue in the long term we can either: a. split off work to another host (unlikely to be viable with ASF infrastructure) b. be more selective about the logs we generate overlaps from (it's already only reporting on nightly logs alone, which is the main one we want) c. change 'report-from-logs' to use more parallelism/background processing (higher load, defeats the purpose of freqsd's batch processing model, more complex code) d. rewrite the overlap-generation code in hit-frequencies to be more efficient (I've already spent a lot of effort on this, it's FAST perl. C maybe? anyway, will take dev effort) e. rewrite 'report-from-logs', 'freqsd' to be more selective about the log sets it generates OVERLAP files from, get the more recent stuff built first. thoughts guys? --j. On Thu, Mar 25, 2010 at 22:06, John Hardin <[email protected]> wrote: > Justin: > > The ruleqa overlap analysis isn't working again, and the rule detail -> SVN > URIs are still pointing at the old rules/trunk sandbox path. > > -- > John Hardin KA7OHZ http://www.impsec.org/~jhardin/ > [email protected] FALaholic #11174 pgpk -a [email protected] > key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C AF76 D822 E6E6 B873 2E79 > ----------------------------------------------------------------------- > Phobias should not be the basis for laws. > ----------------------------------------------------------------------- > 167 days since President Obama won the Nobel "Not George W. Bush" prize > > -- --j.
