Interesting point about the size of the spam db. I am not in front of
the machine now, but I seem to remember that the spam db ADDs to
itself each rebuild. Is there a way to have it build to a point and
then remove old data?
thanks
On 5/7/10, Thomas Eckardt <thomas.ecka...@thockar.com> wrote:
> >When assp starts it's using about 780mb of ram. After running for a
> >couple of days, it gets up to 1.4gb. Is that normal?
>
>
> Depending on the size of your hashes and list, this could be normal - and
> as long there is all running well, there is nothing to change.
> For example: I run 5 Workerthreads, and I use a DB for all hashes and
> lists (also the internal). ASSP starts at 300MB and never goes higher than
> 450MB (even on high load systems) - if the rebuild is used, additional 100
> MB would be needed.
>
> I think, your resulting Spamdb with 1,774,986 records is too large. A good
> size would be between 300k and 400k records.
>
>
> Thomas
>
>
>
> Von: K Post <nntp.p...@gmail.com>
> An: ASSP development mailing list <assp-test@lists.sourceforge.net>
>
> Datum: 06.05.2010 18:06
>
> Betreff: Re: [Assp-test] Info logged about sigoff and sigon
>
>
>
> No databases (yet). All flat files.
>
> Seems crazy that it's 3-5 minutes. Obviously there's something wrong,
> I just don't know what (and I assume on my end).
>
> I'm going to try to set this up on a separate machine for testing with
> the same data to see if that helps. However, it'll take me a couple
> of days to have the time.
>
> When assp starts it's using about 780mb of ram. After running for a
> couple of days, it gets up to 1.4gb. Is that normal?
>
>
> On Wed, May 5, 2010 at 11:56 AM, Thomas Eckardt
> <thomas.ecka...@thockar.com> wrote:
> >> May-01-10 05:43:15 Resulting file 'c:/assp/spamdb.rb.tmp' is 3,908,453
> > bytes
> >> May-01-10 05:47:53 Bayesian Pairs: 165,592 in new mail, 1,774,986 now
> in
> > list
> >
> > The only thing that is done between both lines is
> >
> > &rb_printlog( "\nResulting file '$main::base/spamdb.rb.tmp' is " .
> > &rb_commify($filesize) . " bytes\n" );
> > my $allpairs = scalar keys %main::Spamdb ;
> > my $text = ($main::ReplaceOldSpamdb) ? 'new' : 'in new mail';
> > &rb_printlog("Bayesian Pairs: " . &rb_commify($addcount) . " $text, "
> > . &rb_commify($allpairs) . " now in list\n");
> > &rb_mlog("Bayesian Pairs: " . &rb_commify($addcount) . " $text, " .
> > &rb_commify($allpairs) . " now in list");
> >
> >
> > scalar keys %main::Spamdb
> >
> > Do you use a DB for that? simply the counting of all records takes 3,5
> > minutes !?
> >
> > Thomas
> >
> >
> >
> >
> >
> > Von: K Post <nntp.p...@gmail.com>
> > An: ASSP development mailing list <assp-test@lists.sourceforge.net>
> > Datum: 05.05.2010 15:42
> > Betreff: Re: [Assp-test] Info logged about sigoff and sigon
> >
> >
> >
> > (bump)
> >
> > Still getting this on and off.
> > Thanks
> >
> > On Sat, May 1, 2010 at 11:31 AM, K Post <nntp.p...@gmail.com> wrote:
> >> This builds on my previous thread "SMTP not responding every night at
> >> the same approximte time"
> >>
> >>
> >> Win32, 2.0.2 (1.0.12) -- I haven't updated yet, but have seen similar
> >> things to this sporatically for a while now.
> >>
> >> Here's a clip from the log:
> >> May-01-10 00:41:33 msg88264-08990 88.199.249.6
> >> <spamm...@somedomain.com> Connection idle for 180 secs - timeout
> >> May-01-10 00:41:48 msg88260-10103 88.199.249.6
> >> <spamm...@somedomain2.com> Connection idle for 180 secs - timeout
> >> May-01-10 00:42:29 Info: Loop in Worker_2 was not active for 189
> seconds
> >> May-01-10 00:42:39 Info: Worker_2 : last sigoff in main,
> >> c:\assp\assp.pl, 9301, main::sigoffTry, 1, , , at 11 0:3345
> >> 1272688425.69521 - 9301
> >> May-01-10 00:43:00 msg88307-04742 [InvalidAddress] 190.166.204.2
> >> <spamm...@thriddomain.com> invalid address rejected:
> >> i...@ourdomain.org
> >> May-01-10 00:43:01 Info: Worker_2 : last sigon in main,
> >> c:\assp\assp.pl, 9303, main::sigonTry, 1, , , at 11 0:3345
> >> 1272688425.69543 - 9303
> >> May-01-10 00:43:01 msg88307-04742 190.166.204.2
> >> <spamm...@thriddomain.com> Message-Score: added 2 for invalid address
> >> i...@ourdomain.org, total score for this message is now 2
> >>
> >> Lines 9301 and 9301 are:
> >> &sigoffTry(__LINE__);
> >> and
> >> &sigonTry(__LINE__);
> >>
> >> I've seen this warning / error before. It almost always seems to be
> >> during rebuilds (which this error was).
> >>
> >> We've rebuilt since this error, but I know where the slowdown is.
> >> Look at this excerpt from the rebuild log.
> >> May-01-10 05:43:15 done - Generating weighted Bayesian tuplets
> >>
> >> May-01-10 05:43:15 Resulting file 'c:/assp/spamdb.rb.tmp' is 3,908,453
> > bytes
> >> May-01-10 05:47:53 Bayesian Pairs: 165,592 in new mail, 1,774,986 now
> in
> > list
> >>
------------------------------------------------------------------------------
_______________________________________________
Assp-test mailing list
Assp-test@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-test
