Re: [SLUG] Opinions sought: Exim vs Sendmail
On Tue, 1 Jul 2003, Oscar Plameras wrote: > From: <[EMAIL PROTECTED]> > > On 30 Jun, Oscar Plameras wrote: > > > The reason is as follows: > > > > > > Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your > allocation) > > >= 4,228Mb * 50 = > > > 202,280MB > > > > A cache isn't a complete copy. You store what you allow room for, and > > fall back to your normal mechanism if the entry isn't in the cache. > > You use LRU typically after the cache fills. > > > > Just a point of clarification: > > Cache is structrured data, or data list, or list, kept in CPU MEMORY > all the time and maybe used by a software to locate other informatioin > or to manipulate information. > > Database is structured data, or data list, or list, kept in DISK STORAGE > and maybe used by a software to locate other information > or to manipulate information. This is simply not true. A cache may be kept on disk, and commonly is. eg Squid caches to disk because the overhead of disk retrieval is less than the overhead of repeating the network retrieval. Also, a database may in some cases be implemented as an in-memory structure, although this is sufficiently unusual that you should always be clear about what you're talking about if you wish to avoid the assumption that it is disk-based. Andrew McNaughton -- No added Sugar. Not tested on animals. May contain traces of Nuts. If irritation occurs, discontinue use. --- Andrew McNaughton In Sydney Working on a Product Recommender System [EMAIL PROTECTED] Mobile: +61 422 753 792 http://staff.scoop.co.nz/andrew/cv.doc -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
From: <[EMAIL PROTECTED]> > On 30 Jun, Oscar Plameras wrote: > > The reason is as follows: > > > > Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your allocation) > >= 4,228Mb * 50 = > > 202,280MB > > A cache isn't a complete copy. You store what you allow room for, and > fall back to your normal mechanism if the entry isn't in the cache. > You use LRU typically after the cache fills. > Just a point of clarification: Cache is structrured data, or data list, or list, kept in CPU MEMORY all the time and maybe used by a software to locate other informatioin or to manipulate information. Database is structured data, or data list, or list, kept in DISK STORAGE and maybe used by a software to locate other information or to manipulate information. http://www.acay.com.au/~oscarp/disclaimer.html http://www.acay.com.au/~oscarp -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
> It's quite straight-forward to implement a DNS cache on linux -- run bind > on the box. Isn't this going in circles? For some reason people wanted > to get the cache off the box. > > Personally I haven't found that removing bind gives a performance benefit > (quite the opposite), but different systems use resources in different > combinations, so YMMV. Yes, Oscar suggested removing BIND, but it doesn't make sense to me. It's not that CPU/disk intensive when being used as a cache, so removing it is more than likely just going to increase mail spool queue times, rather than alleviate resources to allow the MTA to work better. If BIND is running as an authoritative DNS server for a very busy domain, it's a different kettle of fish. But I couldn't imagine that being the case from the original email. - Jeff -- Get Informed: SCO vs. IBMhttp://sco.iwethey.org/ http://www.xach.com/debian-users-are-beatniks.html -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On 1 Jul, Andrew McNaughton wrote: > It's quite straight-forward to implement a DNS cache on linux -- run bind > on the box. Isn't this going in circles? For some reason people wanted > to get the cache off the box. > > Personally I haven't found that removing bind gives a performance benefit > (quite the opposite), but different systems use resources in different > combinations, so YMMV. Actually, I gather that Linux's implementation of some part of the email address handling (perhaps the DNS lookups or the reverse lookups or something), was orders of magnitude more inefficient than the same handling under Solaris. If you're interested, I can get more precise info from work tomorrow. luke -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, 30 Jun 2003 [EMAIL PROTECTED] wrote: > On 30 Jun, Oscar Plameras wrote: > > The reason is as follows: > > > > Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your allocation) > >= 4,228Mb * 50 = > > 202,280MB > > A cache isn't a complete copy. You store what you allow room for, and > fall back to your normal mechanism if the entry isn't in the cache. > You use LRU typically after the cache fills. > > This is all very standard stuff, and it's the technique that Solaris > uses to get good performance. So I can't see why Linux couldn't do the > same. It's quite straight-forward to implement a DNS cache on linux -- run bind on the box. Isn't this going in circles? For some reason people wanted to get the cache off the box. Personally I haven't found that removing bind gives a performance benefit (quite the opposite), but different systems use resources in different combinations, so YMMV. Andrew -- No added Sugar. Not tested on animals. May contain traces of nuts. If irritation occurs, discontinue use. --- Andrew McNaughton In Sydney Working on a Product Recommender System [EMAIL PROTECTED] Mobile: +61 422 753 792 http://staff.scoop.co.nz/andrew/cv.doc -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On 30 Jun, Oscar Plameras wrote: > The reason is as follows: > > Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your allocation) >= 4,228Mb * 50 = > 202,280MB A cache isn't a complete copy. You store what you allow room for, and fall back to your normal mechanism if the entry isn't in the cache. You use LRU typically after the cache fills. This is all very standard stuff, and it's the technique that Solaris uses to get good performance. So I can't see why Linux couldn't do the same. The only trick I see is invalidating entries in the cache that are continually being used, during a period when its address changes. The solution would seem to be simply to invalidate the old entry when the DNS server is told of the address change in the normal course of events. luke -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
G'day... Just because I'm being pedantic... Oscar> Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your allocation) Oscar> = 4,228Mb * 50 = Oscar> 202,280MB Oscar> Oscar> Number of IPV6 addresses = we can only imagine this number Number of IPv4 addresses is 2^32 = 4294967296 Allocation per IP address is 32 bits = 4 bytes This *excludes* actual FQDN - if we allow 128 characters for FQDN mapping (a more realistic limit) = 128 bytes (Got to be honest, I'm not sure if there is a limit on a domain name length or what it is.) Total bytes consumed per IP/FQDN mapping = 128 + 4 = 132 Minimum space required = 4294967296 * 132 bytes = 137438953472 bytes = 134217728 KB = 131072 MB = 128 GB However, we are more likely going to need to allow 256 bytes not 128 for each mapping (maximum line length) so total byte consumption per IP/FQDN mapping will be 260 bytes. Minimum space required = 4294967296 * 260 bytes = 1116691496960 bytes = 1090519040 KB = 1064960 MB = 1040 GB Now for IPv6 uses a 128 bit address space (IPv4 uses 32, so 96 more bits making 2^96 more addresses available) Number of IPv6 addresses is 2^128 = 3.4028236692093846346337460743177e+38 ... I don't think I need to continue... Btw... we can imagine 2^128 (it's easiest to remember it in that form...) ... and for that matter all numbers *are* imagined, not real... Numbers are imaginary constructs to describe quantity and don't actually exist (ie... You can see 4 ducks, but you can't see a 4, only a representation of it.) Warmest regards Mike --- Michael S. E. Kraus Network Administrator Capital Holdings Group (NSW) Pty Ltd p: (02) 9955 8000 "Oscar Plameras" <[EMAIL PROTECTED]> Sent by: [EMAIL PROTECTED] 30/06/2003 02:52 PM To: <[EMAIL PROTECTED]> cc: Sydney Linux Users Group <[EMAIL PROTECTED]> Subject: Re: [SLUG] Opinions sought: Exim vs Sendmail From: <[EMAIL PROTECTED]> > On 29 Jun, Oscar Plameras wrote: > > Ideally, one would want all list to be stored in the local Memory but we > > know this is impossible and with the internet growing in leaps and bounds > > the list is growing bigger and faster by the day. Also, you would want a > > DNS software that predicts the information that will be requested just in > > time when it is required. Again this is a mammoth task and out there our > > technical friends have been trying. > > Well, I can't see *any* difference between this problem and the > classical caching problem. Your traffic typically has some coherency > simply because communications tend to be between people who are in some > kind of dialogue. > > It seems to me that the cost of storing an IP address as a string, plus > a word for the decimal IP address, should cost roughly 50 bytes. I.e. > I'd guess you should be able to cache about 20,000 addresses / Mb. I'd > be surprised if any but very large organisations would receive email > from more than that number of *domains* per day. > The reason why it is impossible to store all list in local CPU Memory concurrently is, first, because of the physical limitations of the hardware under the current state of technologies. The reason is as follows: Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your allocation) = 4,228Mb * 50 = 202,280MB Number of IPV6 addresses = we can only imagine this number If you have such a list, imagine the amount of cpu time required to search such a list every time an address is to be found. This is one reason why DNS BIND adopted its methodology and strategy. It is meant to prevent having a list that enlarges to such a huge list with out a way to control. The methodology and strategy is a compromise. And the Sysadmin decides how much to compromise by way of manipulating the configuration. Another reason why there is this limitation is that the complete list is scattered among DNS servers all across the Internet at any given time, the list changes every minute (names change, addresses change, addresses removed, addresses added and so on) and that a local DNS only knows about those addresses previously queried for which this local DNS and its authoritative DNS are answerable. If an address was not previously queried it will not get included in the cache. A single name change will instantaneously make a local list inconsistent with reality. And there are hundreds, perhaps thousands of changes, additions, and removals every minute. In
Re: [SLUG] Opinions sought: Exim vs Sendmail
From: <[EMAIL PROTECTED]> > On 29 Jun, Oscar Plameras wrote: > > Ideally, one would want all list to be stored in the local Memory but we > > know this is impossible and with the internet growing in leaps and bounds > > the list is growing bigger and faster by the day. Also, you would want a > > DNS software that predicts the information that will be requested just in > > time when it is required. Again this is a mammoth task and out there our > > technical friends have been trying. > > Well, I can't see *any* difference between this problem and the > classical caching problem. Your traffic typically has some coherency > simply because communications tend to be between people who are in some > kind of dialogue. > > It seems to me that the cost of storing an IP address as a string, plus > a word for the decimal IP address, should cost roughly 50 bytes. I.e. > I'd guess you should be able to cache about 20,000 addresses / Mb. I'd > be surprised if any but very large organisations would receive email > from more than that number of *domains* per day. > The reason why it is impossible to store all list in local CPU Memory concurrently is, first, because of the physical limitations of the hardware under the current state of technologies. The reason is as follows: Number of IPV4 addresses = 255*255*255*255 * 50 bytes (your allocation) = 4,228Mb * 50 = 202,280MB Number of IPV6 addresses = we can only imagine this number If you have such a list, imagine the amount of cpu time required to search such a list every time an address is to be found. This is one reason why DNS BIND adopted its methodology and strategy. It is meant to prevent having a list that enlarges to such a huge list with out a way to control. The methodology and strategy is a compromise. And the Sysadmin decides how much to compromise by way of manipulating the configuration. Another reason why there is this limitation is that the complete list is scattered among DNS servers all across the Internet at any given time, the list changes every minute (names change, addresses change, addresses removed, addresses added and so on) and that a local DNS only knows about those addresses previously queried for which this local DNS and its authoritative DNS are answerable. If an address was not previously queried it will not get included in the cache. A single name change will instantaneously make a local list inconsistent with reality. And there are hundreds, perhaps thousands of changes, additions, and removals every minute. Incidentally, this is the reason why, when you stop and start a DNS server it takes a while for network throughput to return to normal depending on the number of clients in the network. The DNS cache, local or authoritative, is refreshed every so often, and is expired every so often so the addresses in cache for more than a period of time gets dropped and so the cache will never have the chance to retain the entire list. > If one cache entry saves you thousands or even just tens of > milliseconds, then setting aside some space would give a speed-up of > at least 3 orders of magnitude. One can tune up the named to a point. Tuning up as you know is a compromise; you win some and you lose some and there is no one-way advantage. http://www.acay.com.au/~oscarp/disclaimer.html http://www.acay.com.au/~oscarp -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On 29 Jun, Oscar Plameras wrote: > Ideally, one would want all list to be stored in the local Memory but we > know this is impossible and with the internet growing in leaps and bounds > the list is growing bigger and faster by the day. Also, you would want a > DNS software that predicts the information that will be requested just in > time when it is required. Again this is a mammoth task and out there our > technical friends have been trying. Well, I can't see *any* difference between this problem and the classical caching problem. Your traffic typically has some coherency simply because communications tend to be between people who are in some kind of dialogue. It seems to me that the cost of storing an IP address as a string, plus a word for the decimal IP address, should cost roughly 50 bytes. I.e. I'd guess you should be able to cache about 20,000 addresses / Mb. I'd be surprised if any but very large organisations would receive email from more than that number of *domains* per day. If one cache entry saves you thousands or even just tens of milliseconds, then setting aside some space would give a speed-up of at least 3 orders of magnitude. Bargain! luke -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
From: <[EMAIL PROTECTED]> > > We had some problems at work with system lockups during periods of high > email delivery load, due to the cost of the DNS lookups - on Linux. If > a few hundred emails arivved for delivery at approximately the same > time, it brought the machine to its knees, and took a long long time to > recover, as more email arrived in the meantime. > DNS BIND is always a difficult problem in terms of performance of an overall network. It is mostly because of the way DNS works and partly because of the i386 and Linux architecture as compared to Sun Solaris as you mentioned. In general DNS uses large chunks of CPU Memory area to store information it needs to do its job, namely to associate a Name with an Address or vice versa. The bigger the CPU Memory the better it becomes. This is not entirely true, though. This is not entirely so because by nature DNS information is scattered across the Internet, is changing all the time, and the best way to complete its job quickly is to have the most used information made ready in some place locally which is a portion of CPU memory and for DNS to predict and fetch just in time the information required. When that particular information is not in the list currently found in CPU memory DNS has to fetch this information from its peer DNS may be located locally and if not fetch from somewhere. The way it does this as we know is, it checks for this information from its peer or the upstream authoritative DNS and if it finds there, well and good the search is over. But if not the search continues on to the next upstream and so on. This fetching involves many conversation between the machines exchanging information. The speed depends on the bandwidth currently available and on the capacity of the authoritative server. This process is as fast as the slowest bandwidth or the slowest server upstream. Now, nobody would want to tie down an application running on a local machine just because it is waiting for the DNS in this same machine to complete its job of fetching information from its peer or upstream. So, if you have a local machine that works hard, do not put a DNS named to run on it. Either the application or DNS or both will suffer in their performance. Ideally, one would want all list to be stored in the local Memory but we know this is impossible and with the internet growing in leaps and bounds the list is growing bigger and faster by the day. Also, you would want a DNS software that predicts the information that will be requested just in time when it is required. Again this is a mammoth task and out there our technical friends have been trying. > As I roughly recall from one of our sysadmin people's explanation, > Solaris apparently doesn't suffer the same way since it caches in memory > the DNS lookup results. > The main difference between Sun Solaris and Linux lies in their architecture. Sun Solaris is engineered to use a number of little cpus with their exclusive memory and other buffer memory areas independent from the main CPU memory on a Sun Platform. Linux on the other hand because it relies on the i386 architecture the general and specialists operations are mainly integrated into a single CPU and the CPU memory areas. Perhaps Sun, IBM, Fujitsu, Hitachi, etc, with their own versions of Linux have modified Linux to perform optimized on their respective platforms. With Sun Solaris for example it has lots of buffer memory that goes with independent CPUs to perform specialists operations like buffering, direct memory access, etc., and therefore takes away the load from the main CPU and main memory. Relatively speaking, a Sun Platform as I see it is a number of i386 in one single box. So, a Sun Solaris with 256MHZ CPU with the some amount of memory is not comparable to a Linux on i386 with 256MHZ CPU even if there is one with the same amount of memory . That is why prices for the same CPU and MEMORY configuration of a box for these two are not also comparable. Apart from the differences in hardware make up, there are differences in terms of 'SMARTS' added to it. Hence, the analogy of comparing apples and oranges apply, or really it does not apply ? > Seems like an obvious thing to do. Any hint that the appropriate Linux > DNS component might be improved in the same way? > It is extremely difficult if not totally impossible to have Linux on the i386 architecture as we know it today to come near a Sun Solaris in performance given that both have comparible configuration in terms of CPU and Memory. http://www.acay.com.au/~oscarp/disclaimer.html http://www.acay.com.au/~oscarp -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
[EMAIL PROTECTED] wrote: On 24 Jun, Oscar Plameras wrote: True, email services is dns service intensive. It demands both forward and reverse lookup services. So, when MTA requires DNS services it is serviced by named in the same box. First, the catch is, named is less important We had some problems at work with system lockups during periods of high email delivery load, due to the cost of the DNS lookups - on Linux. If a few hundred emails arivved for delivery at approximately the same time, it brought the machine to its knees, and took a long long time to recover, as more email arrived in the meantime. As I roughly recall from one of our sysadmin people's explanation, Solaris apparently doesn't suffer the same way since it caches in memory the DNS lookup results. Seems like an obvious thing to do. Any hint that the appropriate Linux DNS component might be improved in the same way? NAME /usr/sbin/nscd - name service cache daemon DESCRIPTION Nscd is a daemon that provides a cache for the most common name service requests. The default configuration file, /etc/nscd.conf, determines the behavior of the cache daemon. See nscd.conf(5). -- Del -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On 24 Jun, Oscar Plameras wrote: > True, email services is dns service intensive. It demands both forward and > reverse lookup services. So, when MTA requires DNS services it is > serviced by named in the same box. First, the catch is, named is less > important We had some problems at work with system lockups during periods of high email delivery load, due to the cost of the DNS lookups - on Linux. If a few hundred emails arivved for delivery at approximately the same time, it brought the machine to its knees, and took a long long time to recover, as more email arrived in the meantime. As I roughly recall from one of our sysadmin people's explanation, Solaris apparently doesn't suffer the same way since it caches in memory the DNS lookup results. Seems like an obvious thing to do. Any hint that the appropriate Linux DNS component might be improved in the same way? luke -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
RE: [SLUG] Opinions sought: Exim vs Sendmail
> -Original Message- > From: Anthony Wood [mailto:[EMAIL PROTECTED] > Sent: Wednesday, 25 June 2003 9:41 AM > To: James Gray > Cc: [EMAIL PROTECTED] > Subject: Re: [SLUG] Opinions sought: Exim vs Sendmail > > > On Tue, Jun 24, 2003 at 04:45:42AM -0400, [EMAIL PROTECTED] wrote: > > > > > -Original Message- > > > From: Dinesh Birlasekaran [mailto:[EMAIL PROTECTED] > > > Sent: Tuesday, 24 June 2003 2:29 PM > > > To: James Gray > > > Subject: FW: [SLUG] Opinions sought: Exim vs Sendmail > > > > > > > > > > > > > > > Hi james, > > > > > > What anti virus are u using? > > > > > > Dinesh. > > > > We wrap NAI's "vscan" into inflex. Inflex handles the > > UUencode/decode, virus scanning and magic number calculation as > > well. We actually look at attached files to determine their content > > rather than rely on file names alone. For instance, if someone > > tries to send a Win32 .exe file by renaming it to bogus.txt or > > something, we calculate "bugus.txt" magic number and it will be > > detected as a potentially dangerous attachment and deleted. Quite > > quick and very effective when coupled with the virus scanner :-) We > > ditch hazardous attachments before running the virus scanner - which > > means we end up scanning a LOT of .zip files. > > Does your scanner unzip them and scan the included packages? > > Does your scanner run zipcracker (PVM enabled :-) to open > passworded zip files? > > cheers, > Woody Yes it opens and recurses any common compressed formats (zip, tar, gz, tar.gz, hqx, arj, etc). As for password protected archives, if it finds one and can't open it, it deletes it and sends a small text-file attachment saying not send password protected compressed archives by e-mail, and to use our corp sftp server instead ;) As for zipcracker - I haven't come across that in a long time. Last I poked around with it, it would only handle passwords up to 4 characters...ot overly useful. I'd be interested in hearing more about it's performance etc. Cheers, James -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Tue, Jun 24, 2003 at 04:45:42AM -0400, [EMAIL PROTECTED] wrote: > > > -Original Message- > > From: Dinesh Birlasekaran [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, 24 June 2003 2:29 PM > > To: James Gray > > Subject: FW: [SLUG] Opinions sought: Exim vs Sendmail > > > > > > > > > > Hi james, > > > > What anti virus are u using? > > > > Dinesh. > > We wrap NAI's "vscan" into inflex. Inflex handles the > UUencode/decode, virus scanning and magic number calculation as > well. We actually look at attached files to determine their content > rather than rely on file names alone. For instance, if someone > tries to send a Win32 .exe file by renaming it to bogus.txt or > something, we calculate "bugus.txt" magic number and it will be > detected as a potentially dangerous attachment and deleted. Quite > quick and very effective when coupled with the virus scanner :-) We > ditch hazardous attachments before running the virus scanner - which > means we end up scanning a LOT of .zip files. Does your scanner unzip them and scan the included packages? Does your scanner run zipcracker (PVM enabled :-) to open passworded zip files? cheers, Woody -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
From: "Jeff Waugh" <[EMAIL PROTECTED]> > On Tue, 2003-06-24 at 04:04, Oscar Plameras wrote: > > It is also a good idea to take the /usr/local/sbin/named > > away to another FreeBSD/Linux box. DNS lookups is > > always a slow process and queues other processes > > particularly during Internet peak hours. > > > > Because the box is mainly Email services, and not > > transaction type application such as financial databases, it > > will help alleviate processing bottle neck by increasing > > the time between disc sync'ing as done by the (syncer) > > process. > > It wouldn't be a good idea to shift DNS away from the email server - > much of what an MTA does is DNS lookups, so it is always a good idea to > have a fast, caching name server on the local machine. Better throughput > through better latency. > > Caching (or authoritative, for that matter, on an unbusy domain) DNS > isn't a horribly expensive process. > True, email services is dns service intensive. It demands both forward and reverse lookup services. So, when MTA requires DNS services it is serviced by named in the same box. First, the catch is, named is less important in this particular box with 0.0/1.6 percent CPU utilisation. Sustained CPU activity is more important for the main task done by spamd as indicated by 78.1/14.7 percent CPU/MEM utilisation. The fact that named has far less activity in this box is perhaps due to spamd being so aggressive and leaves little time for other processes to spawn when it should. By moving it to another box within the same network may make both named and spamd happier and livelier. Secondly, with named servicing the MTA in the box, it also answers service request from other services outside this particular box. It is that time that the DNS in this MTA box spends servicing others that we do not wish to loss in view of the capacity requirements for CPU utilisation by spamd. The advantage of named being in the same box as the MTA is negated and lossing more. It sounds to me like this network is rather busy with a number of domains and users being served. 60Gb of email data in a day alone tells the story. -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Tue, 2003-06-24 at 04:04, Oscar Plameras wrote: > It is also a good idea to take the /usr/local/sbin/named > away to another FreeBSD/Linux box. DNS lookups is > always a slow process and queues other processes > particularly during Internet peak hours. > > Because the box is mainly Email services, and not > transaction type application such as financial databases, it > will help alleviate processing bottle neck by increasing > the time between disc sync'ing as done by the (syncer) > process. It wouldn't be a good idea to shift DNS away from the email server - much of what an MTA does is DNS lookups, so it is always a good idea to have a fast, caching name server on the local machine. Better throughput through better latency. Caching (or authoritative, for that matter, on an unbusy domain) DNS isn't a horribly expensive process. - Jeff -- linux.conf.au 2004: Adelaide, Australia http://lca2004.linux.org.au/ "Stick to your guns and get it right." - Havoc Pennington -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
RE: [SLUG] Opinions sought: Exim vs Sendmail
> -Original Message- > From: Dinesh Birlasekaran [mailto:[EMAIL PROTECTED] > Sent: Tuesday, 24 June 2003 2:29 PM > To: James Gray > Subject: FW: [SLUG] Opinions sought: Exim vs Sendmail > > > > > Hi james, > > What anti virus are u using? > > Dinesh. We wrap NAI's "vscan" into inflex. Inflex handles the UUencode/decode, virus scanning and magic number calculation as well. We actually look at attached files to determine their content rather than rely on file names alone. For instance, if someone tries to send a Win32 .exe file by renaming it to bogus.txt or something, we calculate "bugus.txt" magic number and it will be detected as a potentially dangerous attachment and deleted. Quite quick and very effective when coupled with the virus scanner :-) We ditch hazardous attachments before running the virus scanner - which means we end up scanning a LOT of .zip files. --James -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
RE: [SLUG] Opinions sought: Exim vs Sendmail
Quoting [EMAIL PROTECTED]:
> >
> >
> > On Mon, Jun 23, 2003 at 11:19:25AM +1000, James Gray wrote:
> >
> > > We are running into problems when we get a flood of
> > messages (>50/minute)
> > > as the whole mail filtering/scanning thing quickly chews up
> > all CPU time
> >
> > Are you running SpamAssassin as a daemon (spamd)? I had this problem
> > when I first setup SpamAssassin because I was using the perl program
> > (spamassassin) to process each message. I changed to using
> > spamc/spamd
> > and it now has negligible impact on the cpu load.
> >
> >
> > Cheers,
> >
> > John
>
> Here's the pertinent details for the system concerned:
> # ps aux | awk '{print $1 "\t" $3 "\t" $4 "\t" $10 "\t" $11}'
> USER%CPU%MEMTIMECOMMAND
> nobody 78.114.70:11.69 /usr/bin/spamd
> root0.0 0.0 0:28.50 (pagedaemon)
> root0.0 0.0 0:05.64 (vmdaemon)
> root0.0 0.0 0:11.99 (bufdaemon)
> root0.0 0.0 0:11.79 (vnlru)
> root0.0 0.0 8:24.99 (syncer)
> root0.0 0.3 0:39.22 /usr/bin/perl
> bind0.0 1.6 39:49.58/usr/local/sbin/named
> root0.0 0.3 1:13.24 sendmail:
> root0.0 9.4 0:03.92 /usr/bin/spamd
> root0.0 0.5 0:00.01 sendmail:
> root0.0 0.5 0:00.01 sendmail:
> root0.0 0.5 0:00.01 sendmail:
> root0.0 0.6 0:00.03 sendmail:
> root0.0 0.5 0:00.01 /usr/bin/spamc
> root0.0 0.6 0:00.01 sendmail:
> root0.0 0.0 0:00.12 (swapper)
> root0.0 0.0 0:25.62 /sbin/init
>
> (non-relevant processes snipped - sshd/csh/sh etc).
>
> Notice the spamd load? This looks a little high to me. But our spam
> rules are huge (the normal rules that come with Spamassassin + 1168
> custom rules). Those custom rules round out to 3504 lines of
> RULE/DESCRIPTION/SCORE. so relatively large. FWIW we're running
> spamassassin 2.55.
>
> I can send anything that might shed more light on the problem
> (sendmail.cf exerpts etc)just let me know :-)
>
Your vmstat will provide additional info, like a
vmstat -n 30. It will be interesting to see what it shows.
Observing your stats:
1. Over 6 Gb of data processed each day.
1. Over 50 emails per minute translates to over 3000/hour
2. Only two processes for /usr/bin/spamd
with very unbalanced load of 78/0% CPU and 15/9% MEM.
3. Only one process for /usr/bin/spamc of 0.0/9.5% CPU/MEM.
Comments:
1. 6 Gb of data a day at 50 emails per minute is roughly
300K per email, assuming 10 hours of work. So, at peak
time your system is processing 15Mb/minute of data and is flat
out. The 15Mb/minute is quite acceptable.
2. Although two /usr/bin/spamd processes are spawned the
processing load is not distributed which appears to me
/usr/bin/spamd is not multi-threading although the second
process is concurrently loaded in memory as shown by 0% CPU
and 9.4% MEM utilisation.
3. In view that 15Mb/minute processing outcome is quite
acceptable even with a /usr/bin/spamd that is not multi-
threading, I would suggest it is time to add more computing
power to your project even whilst you contemplate re-arranging
your software.
4. There are two ways to add more computing power:
a. Increase the CPU power in a single box which is straight
forward but is limited in its scalability, or
b. Re-arrange the way your applications will compute by
utilising the Network as your Computer. By this I mean
utilising more than one computer connected on a network
to cooperate together to do the job. This is especially
an excellent strategy with Email processing and is
normally done as we scale up our load. This is excellent
solution I believe because it is extremely scalable for
email types of applications. We can quickly scale up or
down by adding or removing a computer/computers in the
work group.
I hope you can resolve your problems soon and hope that
in the process you have fun.
http://www.acay.com.au/~oscarp/disclaimer.html
-
This mail sent through IMP: http://horde.org/imp/
--
SLUG - Sydney Linux User's Group - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
From: "Oscar Plameras" <[EMAIL PROTECTED]>
> > Here's the pertinent details for the system concerned:
> > # ps aux | awk '{print $1 "\t" $3 "\t" $4 "\t" $10 "\t" $11}'
> > USER%CPU%MEMTIMECOMMAND
> > nobody 78.114.70:11.69 /usr/bin/spamd
> > root0.0 0.0 0:28.50 (pagedaemon)
> > root0.0 0.0 0:05.64 (vmdaemon)
> > root0.0 0.0 0:11.99 (bufdaemon)
> > root0.0 0.0 0:11.79 (vnlru)
> > root0.0 0.0 8:24.99 (syncer)
> > root0.0 0.3 0:39.22 /usr/bin/perl
> > bind0.0 1.6 39:49.58/usr/local/sbin/named
> > root0.0 0.3 1:13.24 sendmail:
> > root0.0 9.4 0:03.92 /usr/bin/spamd
> > root0.0 0.5 0:00.01 sendmail:
> > root0.0 0.5 0:00.01 sendmail:
> > root0.0 0.5 0:00.01 sendmail:
> > root0.0 0.6 0:00.03 sendmail:
> > root0.0 0.5 0:00.01 /usr/bin/spamc
> > root0.0 0.6 0:00.01 sendmail:
> > root0.0 0.0 0:00.12 (swapper)
> > root0.0 0.0 0:25.62 /sbin/init
> >
>
It is also a good idea to take the /usr/local/sbin/named
away to another FreeBSD/Linux box. DNS lookups is
always a slow process and queues other processes
particularly during Internet peak hours.
Because the box is mainly Email services, and not
transaction type application such as financial databases, it
will help alleviate processing bottle neck by increasing
the time between disc sync'ing as done by the (syncer)
process.
http://www.acay.com.au/~oscarp/disclaimer.html
--
SLUG - Sydney Linux User's Group - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
RE: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, 23 Jun 2003 [EMAIL PROTECTED] wrote: > > -Original Message- > > From: Andrew McNaughton [mailto:[EMAIL PROTECTED] > > Sent: Monday, 23 June 2003 5:03 PM > > To: James Gray > > Subject: Re: [SLUG] Opinions sought: Exim vs Sendmail > > > > > > > > It sounds to me like you might want to allow more concurrent processes > > than you are at present. Also, the main resource you need more of is > > going to be memory. Have your bean counters taken in that an > > extra 512MB > > is really petty cash level expenditure? In any case, is > > there really any > > problem with allowing more concurrent accesses? > > > > I agree with the suggestion about running your mail filter as > > a daemon if > > possible. If this serializes your filtering then it should > > help a great > > deal. > > > > Andrew > > Spamassassin is already running as a daemon and chews up about 24Mb RAM > just for the parent process - that's almost 10% of our physical RAM!. > spamd children (according to vmstat + ps + top) are all reporting > similar usage (23-27Mb RAM). As you suggest the lack of RAM is killing Most of that memory should be shared, so it's probably not quite as bad as you're suggesting? > us. I did some burst load testing on it yesterday and found that > without limiting child processes for spamd it only took 15 messages in > under 5 seconds to sent system load to 15! 14 children of spamd caused > so much paging that the system ground to a halt (load peaked at 19!!). I don't know the ins and outs of how the spamassassin daemon does work, but this is not how you'd want it to work. The daemon should limit the number of children operating at any given time, with enough parallel requests so that you're system has stuff to do during waits for remote dns and checksum checks, but you don't load up on processes all competing for CPU. > So I did some quick calculations and decided 3 spamd children per CPU > (with 256Mb RAM) would be appropriate (given average time per message, > RAM, other process requirements etc). I managed to send 50 messages in > 7 seconds and system load hit 15 for less than 3 seconds and then > quickly returned to <1. Paging was non-existent and no connections were > refused. So startup scripts are now "spamd -m 3." Sounds better. You've got a fundamental limit on the ammount of messages you can process in a given time - each message takes a given ammount of CPU time, which gives you a pretty good idea of how many messages you can process in a given block of time. At some level of CPU activity you would ideally want to change strategy: stop doing mail filtering while the remote MTA is connected and start putting mail into a spool. You'd want this spool to be processed as resources are available, and mail either delivered or bounced accordingly. A bounce isn't as good as giving an error while you've got the remote MTA still connected, but it allows you to process things at your leisure. This delayed processing has other advantages also. Spam checksum systems take a short while to get spam reports, so delaying processing can mean improved results. I've been thinking of doing a procmail based solution to this on my own mailbox somewhat after delivery. How you'd set up this spooling I'm not sure. I would very much like to hear about such arrangements using exim or postfix. Andrew -- No added Sugar. Not tested on animals. If irritation occurs, discontinue use. --- Andrew McNaughton In Sydney Working on a Product Recommender System [EMAIL PROTECTED] Mobile: +61 422 753 792 http://staff.scoop.co.nz/andrew/cv.doc -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
From: <[EMAIL PROTECTED]>
> >
> >
> > On Mon, Jun 23, 2003 at 11:19:25AM +1000, James Gray wrote:
> >
> > > We are running into problems when we get a flood of
> > messages (>50/minute)
> > > as the whole mail filtering/scanning thing quickly chews up
> > all CPU time
> >
> > Are you running SpamAssassin as a daemon (spamd)? I had this problem
> > when I first setup SpamAssassin because I was using the perl program
> > (spamassassin) to process each message. I changed to using
> > spamc/spamd
> > and it now has negligible impact on the cpu load.
> >
> >
> > Cheers,
> >
> > John
>
> Here's the pertinent details for the system concerned:
> # ps aux | awk '{print $1 "\t" $3 "\t" $4 "\t" $10 "\t" $11}'
> USER%CPU%MEMTIMECOMMAND
> nobody 78.114.70:11.69 /usr/bin/spamd
> root0.0 0.0 0:28.50 (pagedaemon)
> root0.0 0.0 0:05.64 (vmdaemon)
> root0.0 0.0 0:11.99 (bufdaemon)
> root0.0 0.0 0:11.79 (vnlru)
> root0.0 0.0 8:24.99 (syncer)
> root0.0 0.3 0:39.22 /usr/bin/perl
> bind0.0 1.6 39:49.58/usr/local/sbin/named
> root0.0 0.3 1:13.24 sendmail:
> root0.0 9.4 0:03.92 /usr/bin/spamd
> root0.0 0.5 0:00.01 sendmail:
> root0.0 0.5 0:00.01 sendmail:
> root0.0 0.5 0:00.01 sendmail:
> root0.0 0.6 0:00.03 sendmail:
> root0.0 0.5 0:00.01 /usr/bin/spamc
> root0.0 0.6 0:00.01 sendmail:
> root0.0 0.0 0:00.12 (swapper)
> root0.0 0.0 0:25.62 /sbin/init
>
> (non-relevant processes snipped - sshd/csh/sh etc).
>
> Notice the spamd load? This looks a little high to me. But our spam
> rules are huge (the normal rules that come with Spamassassin + 1168
> custom rules). Those custom rules round out to 3504 lines of
> RULE/DESCRIPTION/SCORE. so relatively large. FWIW we're running
> spamassassin 2.55.
>
> I can send anything that might shed more light on the problem
> (sendmail.cf exerpts etc)just let me know :-)
>
Observations from your stats:
1. Over 6Gb of data, 10,000+ of emails, and 50+ emails/minute during
peak hours.
2. Two instances of /usr/bin/spamd with 78.1/14.7 CPU/MEM on
one and 0.0/9.4 CPU/MEM on the second. One instance of
/usr/bin/spamc is 0.0/0.5 CPU/MEM.
3. Five instances of sendmail with not one exceeding 0.0/0.6
CPU/MEM.
4. Swapper with almost not activity.
Comments
1. It appears that your system is processing at an average and
sustained rate of 30Mb per minute. This is quite good throughput.
2. I can see that /usr/bin/spamd is not multi-threading nicely as
one instance is extremely active and aggressive whilst the
second instance is not processing anything although it is
concurrently loaded in memory.
3. Sendmail MTA which you have identified to replace are
not actually chewing up too much resources as indicated by
five idle processes.
4. Physical memory is sufficient as indicated by the Swapper
process having done almost nothing.
5. In view of the above, I would suggest that Computing
Power is less than adequate for the amount of work that you
have at this moment and it is not so much your current
mail and accessories software that are directly the cause
for this inadequacy.
6. In view of the above, I would suggest further that there
are two ways to resolve your situation, namely:
a. Increase the CPU power of your box even whilst you
contemplate replacing or re-arranging your software.
This is the simplest and straight forward solution as
you know.
b. The other solution is, to use your Network as your
Computer. By this, I mean use two or several computers
connected by a networking technology such as TCP/IP
to provide computer processing to do a job. This is
especially useful for applications like Email Services
as it gives you flexibility and scalability as you go upwards
or downwards with your load. You can add or remove
computer/computers in your Network Computer
depending on your requirements. There is little that you
have to change in your network except for the
re-arrangement of your BIND configuration specifically
your MX records.
With your stats, it might also help if you have a look at
your vmstat, like
vmstat -n 30
I hope you will be able to alleviate your problems
quickly and in the process have some fun.
http://www.acay.com.au/~oscarp/disclaimer.html
--
SLUG - Sydney Linux User's Group - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
RE: [SLUG] Opinions sought: Exim vs Sendmail
> -Original Message- > From: Andrew McNaughton [mailto:[EMAIL PROTECTED] > Sent: Monday, 23 June 2003 5:03 PM > To: James Gray > Subject: Re: [SLUG] Opinions sought: Exim vs Sendmail > > > > It sounds to me like you might want to allow more concurrent processes > than you are at present. Also, the main resource you need more of is > going to be memory. Have your bean counters taken in that an > extra 512MB > is really petty cash level expenditure? In any case, is > there really any > problem with allowing more concurrent accesses? > > I agree with the suggestion about running your mail filter as > a daemon if > possible. If this serializes your filtering then it should > help a great > deal. > > Andrew Spamassassin is already running as a daemon and chews up about 24Mb RAM just for the parent process - that's almost 10% of our physical RAM!. spamd children (according to vmstat + ps + top) are all reporting similar usage (23-27Mb RAM). As you suggest the lack of RAM is killing us. I did some burst load testing on it yesterday and found that without limiting child processes for spamd it only took 15 messages in under 5 seconds to sent system load to 15! 14 children of spamd caused so much paging that the system ground to a halt (load peaked at 19!!). So I did some quick calculations and decided 3 spamd children per CPU (with 256Mb RAM) would be appropriate (given average time per message, RAM, other process requirements etc). I managed to send 50 messages in 7 seconds and system load hit 15 for less than 3 seconds and then quickly returned to <1. Paging was non-existent and no connections were refused. So startup scripts are now "spamd -m 3." So I've put in a capex request for an additional 512MB stick of RAM and see where that gets us. If I get the RAM, I'll increase the number of spamd children to 6 or 7. I've also put in a request for a 2nd CPU but on a different capex :P If I'm *REALLY* lucky, I might get both.or maybe not too. Thanks to all for the pointers. Looks like I might be able to tune perl and spamassassin a little more to get a bit more out of it :-) --James -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, 2003-06-23 at 02:50, Tony Green wrote: > On Mon, 2003-06-23 at 11:43, Bruce Badger wrote: > > Could you expand a bit on why postfix over exim? > > I've heard a LOT of support for postfix and have seen some diehard > sendmail guys switch over. Postfix was designed as a drop-in replacement for sendmail, so it's an easier upgrade path. Whilst Exim is very well documented, it's too baroque for my tastes, and *very* dynamic. That's good if you want it, but if you just want straightforward, secure, modular, yada, you're basically choosing between Qmail and Postfix. And Postfix has the leg up on straightforwardness over Qmail too. :-) > I've not heard of many exim installs in production, I know a number of > people use it for 'small' (home) installs. There are a few reasonably high profile sites running on Exim, mostly due to its dynamic stuff. - Jeff -- Fear and Loathing in Dublin, Ireland -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, 2003-06-23 at 02:19, James Gray wrote: > Are there other alternatives (Postfix, Qmail etc) that people have experience with? > We currently pump 10,000+ messages and 6Gbytes+ a day through our mail gateway. Apart from my usual recommendations for Postfix (easy to configure, robust, secure, yada yada), I'd recommend it in this case because it's really easy to configure resources per queue or transport, which sounds like what you need in this case. ie. you can give it a maximum number of mails that it can send through your virus scanning transport, and a different number to accept externally. That means you can accept a lot of mail, and only queue it up to the virus scanner at a rate that the machine can handle. I'm sure other MTAs will do the same thing, I just find Postfix to have a really good simplicity/minimalism/features/power sweet spot. :-) - Jeff -- Fear and Loathing in Dublin, Ireland -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
I use Exim on several systems that handle multiple domains (5-15) and about 50-150 users it goes through a considerable amount of mail (I'll dig up some specs if anyone is interested) and have never really had a problem. I have had a few test installs doing LDAP lookups for authentication etc. I have found it flexible and easy to configure. ---Gareth Walters -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
RE: [SLUG] Opinions sought: Exim vs Sendmail
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]
> Behalf Of John Clarke
> Sent: Monday, 23 June 2003 11:40 AM
> To: [EMAIL PROTECTED]
> Subject: Re: [SLUG] Opinions sought: Exim vs Sendmail
>
>
> On Mon, Jun 23, 2003 at 11:19:25AM +1000, James Gray wrote:
>
> > We are running into problems when we get a flood of
> messages (>50/minute)
> > as the whole mail filtering/scanning thing quickly chews up
> all CPU time
>
> Are you running SpamAssassin as a daemon (spamd)? I had this problem
> when I first setup SpamAssassin because I was using the perl program
> (spamassassin) to process each message. I changed to using
> spamc/spamd
> and it now has negligible impact on the cpu load.
>
>
> Cheers,
>
> John
Here's the pertinent details for the system concerned:
# ps aux | awk '{print $1 "\t" $3 "\t" $4 "\t" $10 "\t" $11}'
USER%CPU%MEMTIMECOMMAND
nobody 78.114.70:11.69 /usr/bin/spamd
root0.0 0.0 0:28.50 (pagedaemon)
root0.0 0.0 0:05.64 (vmdaemon)
root0.0 0.0 0:11.99 (bufdaemon)
root0.0 0.0 0:11.79 (vnlru)
root0.0 0.0 8:24.99 (syncer)
root0.0 0.3 0:39.22 /usr/bin/perl
bind0.0 1.6 39:49.58/usr/local/sbin/named
root0.0 0.3 1:13.24 sendmail:
root0.0 9.4 0:03.92 /usr/bin/spamd
root0.0 0.5 0:00.01 sendmail:
root0.0 0.5 0:00.01 sendmail:
root0.0 0.5 0:00.01 sendmail:
root0.0 0.6 0:00.03 sendmail:
root0.0 0.5 0:00.01 /usr/bin/spamc
root0.0 0.6 0:00.01 sendmail:
root0.0 0.0 0:00.12 (swapper)
root0.0 0.0 0:25.62 /sbin/init
(non-relevant processes snipped - sshd/csh/sh etc).
Notice the spamd load? This looks a little high to me. But our spam
rules are huge (the normal rules that come with Spamassassin + 1168
custom rules). Those custom rules round out to 3504 lines of
RULE/DESCRIPTION/SCORE. so relatively large. FWIW we're running
spamassassin 2.55.
I can send anything that might shed more light on the problem
(sendmail.cf exerpts etc)just let me know :-)
Cheers,
James
--
SLUG - Sydney Linux User's Group - http://slug.org.au/
More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, 2003-06-23 at 11:43, Bruce Badger wrote: > Could you expand a bit on why postfix over exim? I've heard a LOT of support for postfix and have seen some diehard sendmail guys switch over. I've not heard of many exim installs in production, I know a number of people use it for 'small' (home) installs. I spent 8 years learning and working with sendmail, I'm not about to move now ;-) Quickly, to the BatBook! Greeno -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On 23 Jun 2003 11:25:36 +1000 "Tony Green" <[EMAIL PROTECTED]> wrote: > I'd also suggest postfix over exim if you're going to migrate away from > sendmail, though others will be able to give you more help there (I'm a > sendmail fan). Could you expand a bit on why postfix over exim? Many thanks. -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, Jun 23, 2003 at 11:19:25AM +1000, James Gray wrote: > We are running into problems when we get a flood of messages (>50/minute) > as the whole mail filtering/scanning thing quickly chews up all CPU time Are you running SpamAssassin as a daemon (spamd)? I had this problem when I first setup SpamAssassin because I was using the perl program (spamassassin) to process each message. I changed to using spamc/spamd and it now has negligible impact on the cpu load. Cheers, John -- whois [EMAIL PROTECTED] GPG key id: 0xD59C360F http://kirriwa.net/john/ -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
Re: [SLUG] Opinions sought: Exim vs Sendmail
On Mon, 2003-06-23 at 11:19, James Gray wrote: > 1. Mail relay for approx 12 domains > 2. Content detection (magic number etc) for DOS/Win32 binaries etc. > 3. Virus scanning (we use NAI VirusScan Enterprise) > 4. Spam filtering (currently SpamAssassin) I do all of this on a small box running sendmail, so I'd check the config of the machine and the apps. I'd also suggest postfix over exim if you're going to migrate away from sendmail, though others will be able to give you more help there (I'm a sendmail fan). Greeno -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
[SLUG] Opinions sought: Exim vs Sendmail
Hi All, Just spec'ing out some stuff with our corporate gateway mail relay. It is currently running FreeBSD 4.6, sendmail, inflex, spamassassin - the usual mail-gateway stuff. It's a Compaq Proliant box with a P3-866 and 256Mb RAM, with a RAID 0/1 (Compaq Smart Array2 IIRC). We are running into problems when we get a flood of messages (>50/minute) as the whole mail filtering/scanning thing quickly chews up all CPU time sending the system load to >15 which causes sendmail to start refusing connections. Obviously this is a problem. We can't up-spec the box due to bean-counter constraints, so I'm looking at replacing this convoluted sendmail+inflex+ripmime+spamassassin with a more elegant Exim+mailscanner (http://www.sng.ecs.soton.ac.uk/mailscanner/). The specific needs we have are: 1. Mail relay for approx 12 domains 2. Content detection (magic number etc) for DOS/Win32 binaries etc. 3. Virus scanning (we use NAI VirusScan Enterprise) 4. Spam filtering (currently SpamAssassin) Would Exim+mailscanner have a lower overhead compared to our current setup? Would Linux (Debian or RedHat are current candidates) provide better task and/or resource management? Are there other alternatives (Postfix, Qmail etc) that people have experience with? We currently pump 10,000+ messages and 6Gbytes+ a day through our mail gateway. Thanks in advance :-) --James -- SLUG - Sydney Linux User's Group - http://slug.org.au/ More Info: http://lists.slug.org.au/listinfo/slug
