Hi!
Short:
>>
>>Upgrade from 1.x to 2.x
>>
>
> For some reason I thought that ASSP 2 required a DB server.
>
Use "DB:" with MySQL / MS SQL ( and BerkeleyDB for other ) is hi recommended,
because
flat files == _terrible_ memory usage
See later:
1) Use "DB:" with MySQL / MS SQL if possible
2) Turn On all options a-la "Use BerkeleyDB"
( a-la P.S.
>
> This is not the case, eh?
> I'm in a company with ~45 employees and processing ~2500 emails/day.
>
See later:
. . . recommend the use of virtualization ( Hyper-V / XEN / ESX / etc.) ,
especially for small businesses:
consolidation costs / better TCO
+
==
. . .
> says 5 workers are active.
Try 7 workers if have free RAM and( or) try RAM-Dirve for TMPDb
Try 3 workers in case small size RAM
. . .
==
+
etc.
)
Full:
>>
>>Upgrade from 1.x to 2.x
>>
>
> For some reason I thought that ASSP 2 required a DB server.
>
Use "DB:" with MySQL / MS SQL ( and BerkeleyDB ( for other) ) is hi
recommended, because
flat files == _terrible_ memory usage
See later:
1) Use "DB:" with MySQL / MS SQL if possible
2) Turn On all options a-la "Use BerkeleyDB"
===
. . .
>> Do you use any DB ?
> No, just flat files. And, I just checked again. it's now running at 1.9GB
> and 112MB swap
flat files == _terrible_ memory usage
This
VVM>>>Win 2003 x86 720Mb for OS + 1Gb for soft => 5 . . 7 workers ( MySQL ( or
other SQL Server) + BerkeleyDB
_absolutely_ requred )
may be read as
If use both MySQL ( "DB:") and BerkeleyDB ( for other) 7 workers ( on x86) use
608 Mb
. . .
==
===
. . .
Thomas Eckardt>>
Thomas Eckardt>> An ASSP system should never swap memory!
Thomas Eckardt>>
Thomas Eckardt>> 1GB at startup - this is only normal (if DB: is used for all
main lists
Thomas Eckardt>> and hashes) for a setup of 8-10 SMTP workers on a 64 Bit Perl.
Thomas Eckardt>>
Thomas Eckardt>> 4Gb(all of the servers memory) - looks like the rebuild task
uses no
Thomas Eckardt>> database for the internal hashes!?
P.K.> All is checked for mysql . . .
See later:
1) Use "DB:" with MySQL / MS SQL if possible
P.K.> You say: "looks like the rebuild task uses no database for the internal
hashes!?"
P.K.> How can I check this?
See later:
2) Turn On all options a-la "Use BerkeleyDB"
. . .
==
{
==
VVM>> 1) Use "DB:" with MySQL / MS SQL if possible
A-la:
myhost:=localhost
DBdriver:=mysql,PORT=3306
mydb:=asspdb
pbdb:=DB:
DelayShowDB:=DB:
DelayShowDBwhite:=DB:
spamdb:=DB:
whitelistdb:=DB:
redlistdb:=DB:
persblackdb:=DB:
delaydb:=DB:
ldaplistdb:=DB:
==
}
+
{
VVM> 2) Turn On all options a-la "Use BerkeleyDB"
==
-----Original Message-----
From: Thomas Eckardt
Sent: Saturday, October 29, 2011 3:31 AM
To: ASSP development mailing list
Subject: [Assp-test] Antwort: Re: Antwort: Re: fixed pre version 2.1.2 11301
Steve, install BerkeleyDB engine and module and set 'useDB4Rebuild' to ON
- will solve your problem.
. . .
==
useBerkeleyDB:=1
griplist:=griplist
useDB4griplist:=1
useDB4IntCache:=1
useDB4Rebuild:=1
}
See more details in P.P.S.
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
----- Original Message -----
From: "Jay Tarbox"
To: "For Users of ASSP"
Sent: Tuesday, September 16, 2014 9:01 PM
Subject: Re: [Assp-user] Upgrade from 1.x to 2.x
> For some reason I thought that ASSP 2 required a DB server. This is not the
> case, eh?
> I'm in a company with ~45 employees and processing ~2500 emails/day.
>
P.P.S.
{{{
----- Original Message -----
From: "Victor Miasnikov"
To: "Doug Lytle"; "ASSP development mailing list"
Sent: Thursday, March 29, 2012 6:53 PM
Subject: Use both MySQL ( "DB:") and BerkeleyDB ( for other) <= flat files ==
_terrible_ memory usage Re: [Assp-test]
Antwort: Re: Antwort: Re: Antwort: Memory footprint
Hi!
>> Do you use any DB ?
> No, just flat files. And, I just checked again. it's now running at 1.9GB
> and 112MB swap
flat files == _terrible_ memory usage
This
VVM>>>Win 2003 x86 720Mb for OS + 1Gb for soft => 5 . . 7 workers ( MySQL ( or
other SQL Server) + BerkeleyDB
_absolutely_ requred )
may be read as
If use both MySQL ( "DB:") and BerkeleyDB ( for other) 7 workers ( on x86) use
608 Mb
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
----- Original Message -----
From: "Victor Miasnikov" vvm
To: "PMW Troy"
Cc: "ASSP development mailing list"
Sent: Friday, March 23, 2012 2:38 PM
Subject: Re: [Assp-test] ASSP V2 Problem Worker_X accept to client failed
. . .
Thomas Eckardt>> - too high setting for 'maxbytes'
"To:" PMW Troy:
maxbytes ? 4Kb? 8? 16? Or?
> says 5 workers are active.
Try 7 workers if have free RAM and( or) try RAM-Dirve for TMPDb
Try 3 workers in case small size RAM
> I will try updating to perl 5.12, and see if that helps.
And even to 5.14 . . . <= in case small size RAM
( Updated 2014-09-17, VVM say: 5.16 , 5.18 , 5.20 )
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
P.S.
Thomas Eckardt>> - bad system design - too less RAM (min 1GB of the non system
part)
Yes: for example: Win 2003 x86 720Mb for OS + 1Gb for soft => 5 . . 7 workers
( MySQL ( or other SQL Server) +
BerkeleyDB _absolutely_ requred )
Thomas Eckardt>> and/or CPU cores (min 4)
PMW Troy:>
PMW Troy> I would suggest adding those minimums to the archive, because this is
PMW Troy> the first I've heard of them, especially the minimum 4 core part.
First of all, more than 1 (one) CPU core
2 core is absolutly(!) minimum
4 core is _recommended_ , but in any case close to minimum
( and see in P.P.P.S. if "use of virtualization ( Hyper-V / XEN / ESX /
etc.)" -- count of CPU is not big problem)
P.P.S.
PMW Troy:> first I've heard . . .
Don't forgot about storage subsystem ( HDD / RAID) :
----- Original Message -----
From: "Thomas Eckardt"
To: "ASSP development mailing list"
Sent: Monday, November 21, 2011 8:44 PM
Subject: [Assp-test] Antwort: best config for rebuildspam and HMM
. . .
There are multiple possible reasons for such a slowness. I think, slow IO
performance is the reason in your case. BerkeleyDB (use for the temporary
HMM DB's) needs a very fast IO system, in case large DB's like HMM are
processed.
. . .
It is not important that the IO system could process many MB/s - IO/s is
much more important.
. . .
My IO system (in an ESX server) for assp is designed as follows.
- LSI high performance SAS-CTL with 256 MB Cache
- 6 disks each with 10Krpm in a RAID10
--
P.P.P.S.
> We're a small business
I strongly recommend the use of virtualization ( Hyper-V / XEN / ESX / etc.) ,
especially for small businesses:
consolidation costs / better TCO
--
----- Original Message -----
From: "Thomas Eckardt"
To: "ASSP development mailing list"
Sent: Wednesday, March 21, 2012 7:11 PM
Subject: [Assp-test] Antwort: ASSP V2 problem Worker_x accept to clientfailed
These are the most common reasons:
- bad system design - too less RAM (min 1GB of the non system part) and/or
CPU cores (min 4)
- the memory of the assp processes is paged
- too slow DNS-servers are used ( fast local DNS servers are recommended)
- a too slow database server is used (seems some MSSQL-setup,drivers....
makes problems because of uncleaned DB-cursors - use MySQL instead)
- regular expressions are looping or taking too long to process
- too high setting for 'maxbytes'
- too less workers (5-7/9) - highload systems with more than 300.000
connection per day could require a special/professional setup and system
design
- Perl modules are not uptodate - Perl 5.12 or higher is recommended
Thomas
}}}
{{{{
----- Original Message -----
From: "Victor Miasnikov"
To: "Pontus Karlsson"; "ASSP development mailing list"
Sent: Tuesday, April 03, 2012 2:46 PM
Subject: [Assp-test] <Any>db:=DB: Fw: 1) Use "DB:" with MySQL / MS SQL
ifpossible 2) Turn On all options a-la "Use
BerkeleyDB" Re:Antwort: Stable versions?
Hi!
Pontus Karlsson>>>I think it has something to do with the fact that ASSP goes
from 1Gb to
Pontus Karlsson>>>4Gb(all of the servers memory) memory consumption in less
than 3 days.
Look like memory leak => try restart assp.pl process every 1 or 2 days by
ReStartSchedule
( Updated 2014-09-17, VVM say:
Best configure restart by MemoryUsageCheckSchedule / MemoryUsageLimit
changelog_2.3.X.txt
==
2013-05-28
. . . in assp 2.3.3 build 13148:
. . .
added:
'MemoryUsageLimit','Memory Limit in MB that ASSP could use'
'The memory limit in megabyte the assp process could use at maximum on your
system. Set this to empty or zero to
disable the feature. The check is done using the schedule defined in
MemoryUsageCheckSchedule .
If the assp process uses more memory than the limit at a scheduled time and
assp is able to restart it self -
a restart will be done within 15 seconds.
. . .
'MemoryUsageCheckSchedule','Schedule(s) to check the ASSP process memory usage
s'
'The schedule(s) that is used to check the current memory usage of the assp
process compared to the MemoryUsageLimit.
Default value is (0-59/10 * * * *), which means every 10 minutes.
. . .
==
)
Thomas Eckardt>>
Thomas Eckardt>> An ASSP system should never swap memory!
Thomas Eckardt>>
Thomas Eckardt>> 1GB at startup - this is only normal (if DB: is used for all
main lists
Thomas Eckardt>> and hashes) for a setup of 8-10 SMTP workers on a 64 Bit Perl.
Thomas Eckardt>>
Thomas Eckardt>> 4Gb(all of the servers memory) - looks like the rebuild task
uses no
Thomas Eckardt>> database for the internal hashes!?
Pontus Karlsson> All is checked for mysql use in the ASSP interface.
Pontus Karlsson> Are there switches elsewhere to set?
See later:
1) Use "DB:" with MySQL / MS SQL if possible
Pontus Karlsson> You say: "looks like the rebuild task uses no database for the
internal hashes!?"
Pontus Karlsson> How can I check this?
See later:
2) Turn On all options a-la "Use BerkeleyDB"
{
==
VVM>> 1) Use "DB:" with MySQL / MS SQL if possible
A-la:
myhost:=localhost
DBdriver:=mysql,PORT=3306
mydb:=asspdb
pbdb:=DB:
DelayShowDB:=DB:
DelayShowDBwhite:=DB:
spamdb:=DB:
whitelistdb:=DB:
redlistdb:=DB:
persblackdb:=DB:
delaydb:=DB:
ldaplistdb:=DB:
==
}
+
{
VVM> 2) Turn On all options a-la "Use BerkeleyDB"
==
-----Original Message-----
From: Thomas Eckardt
Sent: Saturday, October 29, 2011 3:31 AM
To: ASSP development mailing list
Subject: [Assp-test] Antwort: Re: Antwort: Re: fixed pre version 2.1.2 11301
Steve, install BerkeleyDB engine and module and set 'useDB4Rebuild' to ON
- will solve your problem.
. . .
==
useBerkeleyDB:=1
griplist:=griplist
useDB4griplist:=1
useDB4IntCache:=1
useDB4Rebuild:=1
}
Best regards, Victor Miasnikov
Blog: http://vvm.blog.tut.by/
-----Original Message-----
From: Thomas Eckardt
Sent: Tuesday, April 03, 2012 9:07 AM
To: p.k.@ZZZZZZZZZZZZZ ; ASSP development mailing list
Subject: [Assp-test] Antwort: Re: Antwort: Stable versions?
>Are there switches elsewhere to set?
- install BerkelyDB and the perl module BerkelyDB
- check 'useDB4Rebuild' and 'useDB4IntCache' and 'useDB4griplist'
- setup MySQL so that it never terminates any connection on timeout !
- setup MySQL to allow at least 800 concurrent connections
- use INNODB for the assp tables
. . .
Thomas
--
-----Original Message-----
From: Thomas Eckardt
Sent: den 3 april 2012 10:17
To: p.k.@ZZZZZZZZZZZZZ ; ASSP development mailing list
Subject: Antwort: [Assp-test] Stable versions?
. . .
>I think it has something to do with the fact that ASSP goes from 1Gb to
>4Gb(all of the servers memory) memory consumption in less than 3 days.
An ASSP system should never swap memory!
1GB at startup - this is only normal (if DB: is used for all main lists
and hashes) for a setup of 8-10 SMTP workers on a 64 Bit Perl.
4Gb(all of the servers memory) - looks like the rebuild task uses no
database for the internal hashes!?
Thomas
--
}}}}
------------------------------------------------------------------------------
Want excitement?
Manually upgrade your production database.
When you want reliability, choose Perforce
Perforce version control. Predictably reliable.
http://pubads.g.doubleclick.net/gampad/clk?id=157508191&iu=/4140/ostg.clktrk
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
