Kevin,
It would be best to bypass your gateway scanning as that would probably
be a bottleneck before Declude. If you could virus scan with Declude,
that would be all the better. Maybe Scott could help out with a
temporary license for testing.
I would be happy to load your server for you. It should be +50 times
less demanding to send unscanned than it is to receive and scan, and I
can dump this stuff into MS SMTP with a fair amount of ease and pound
you to death :) A simple VBScript that writes a file to the Drop
directory would do the trick, and I could use one that has an
attachment for testing the virus scanners in combination, and also one
without to test something that is more purely Declude. I could also
probably randomize a Received header IP in order to pound on the DNSBL
lookups.
BTW, this seems to be about the same setup as we have going, but we
went with RAID 5 for the 6 drive setup. As a result of my research, I
concluded that RAID 5 across 6 drives was better than RAID 0 across 3
drives (RAID 10 with 6) when it comes to writes, and it is almost twice
as fast on reads. We put an LSI Megaraid 320-2 with 256 MB of battery
backed up writeback cache in as the controller, and the drive lights
are amazingly inactive. We do over 100k messages a day and we were
storing every blocked spam in at least one E-mail account (extra write
in a massive file) along with other I/O wasteful stuff like gatewaying
through MS SMTP on the same box so that we can do envelope address
validation which means an extra write for each message on top of IMail
and Declude.
I'm confident that RAID 5 will provide us with enough I/O to outlast
the processors under this configuration, and despite the popular
belief, RAID 5 is generally better than RAID 10 if you have a fixed
number of hard drives that you can use. IMO, people get confused when
it comes to comparing RAID performance with a fixed number of drives
dedicated to an array, and a fixed number of drives total. It appears
from the benchmarks that I came across in research that 6 disk RAID 5
performs about as well as 3.5 - 4 disk RAID 0 on writes, and about the
same as 5 disk RAID 0 on reads, so for the same performance in RAID 10,
you would need 7 - 8 disks to match the performance of 6 disks in RAID
5 on writes, and 10 disks on reads. You can improve things further
because RAID 5 gives you more capacity and you can increase I/O by not
claiming the entire capacity for an array, essentially using just the
outer surfaces of the disks which are the fastest to travel across the
heads, and the heads have to travel less across the disks. We have
already had our new server pegged twice at 100% for 30 minutes at a
time due to floods from a single automated source, and the redline was
absolutely perfect meaning that there were no issues with disk
contention.
As a side note, it appears that performance starts to level off with
RAID 5 at around 6 drives. This is probably due to not being able to
split files across all of the hard drives effectively due to the stripe
size. It would then seem logical that RAID 50 would be better to use
if you have 8 or more drives in an array, splitting the spanning part
of the array across 2 different channels or even cards for the best
result.
Matt
Kevin Bilbee wrote:
We are getting a new server this week or next
week. I could do a configuration with HT on ad with HT off and if we
could find some user to load the server after our business hours we
could do come real testing on a server not setup for production.
It is going to be a dual zeon 3.0 with 1 gig of
ram. We will have 6 drives 3 mirrors, system, spool and log, mailboxes.
Anyone willing to help load the server after our
business hours?
Matt what configuration would you like to see??
We only run declude with iamil and our virus scanning is done on our
gateways.
Kevin Bilbee
Kevin,
That's a good question and one that I was wondering about myself.
Sniffer isn't hyperthreaded, but by it's nature, that has no ill
effects on the application, although it would perform better with it
off. It is just simply lean enough that it shouldn't be maxing out a
processor before the bulk of Declude and IMail does. Maybe with
persistent Sniffer it would make some sense to make it multi-processor
capable, but on the flipside, the leanness of the application might add
more overhead than benefit when made multi-processor aware (I have no
idea where the point is that a benefit would be found). On a dual 3
GHz hyperthreaded system, Sniffer's single thread is undetectable, and
on a dual 1 GHz system run to capacity, it was barely noticeable when
using the persistent mode.
With Declude there is more opportunity for refinement due to the
overhead in some configurations (especially custom filters). Declude
is multi-processor capable and it can use a lot of processor over
periods of several seconds as it goes from virus scanning to spam
blocking to message handling. The big determining factor probably has
more to do with how many processes might be running at one time. On my
system we typically have 5-10 processes running simultaneously during
peak hours, and I'm thinking that the number of processes should play
into the determination. Much of the CPU overhead comes with the virus
scanning, especially if you are running two scanners, and it appears
that this stuff is also multi-processor capable. So it seems like the
norm for this stuff, but I am not sure if it would be better or worse
with it off.
I just wanted to add some thoughts of mine that would play into the
logic for someone with the experience that would know. I could
probably do a stress test at some point to get a better idea, but that
would be a big production considering the distance to my server and the
hours that I would need to do this at. Maybe we have some other
volunteers :) I'm thinking that a 5,000 message load with a scannable
attachment dropped into a MS SMTP instance to relay off of
IMail/Declude tried both ways ought to show the difference.
Matt
Kevin Bilbee wrote:
Speaking of hyperthreading has anyone checked the performance of
Imail/declude with hyperthreading turned on and off. I know some apps have
showed better perforamnce with hyperthreading turned off.
Kevin Bilbee
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]On Behalf Of Sanford Whiteman
Sent: Saturday, October 09, 2004 3:22 PM
To: Keith Johnson
Subject: Re[4]: [Declude.JunkMail] Filter file maintenance suggestion
What type of CPU overhead do you experience with running SA/SPAMD
with Declude? I saw the tech doc and it mentioned that it takes up
20MB of memory for each config file load in serial.
Memory, shmemory. Pretty cheap stuff. :)
Are you aware of anyone running this on machines pulling over 200K
emails each day?
Yes, but not on _mailbox servers_. These are dedicated relay/scanning
boxes, purpose built with RAID 0+1 and two (four w/hyperthreading)
CPUs. You have to know what you're building for.
--Sandy
------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
e-mail: [EMAIL PROTECTED]
SpamAssassin plugs into Declude!
http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release
/
Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail
Aliases!
http://www.mailmage.com/products/software/freeutils/exchange2aliases/downl
oad/release/
http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/
release/
---
[This E-mail was scanned for viruses by Declude Virus
(http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
[This E-mail was scanned for viruses by Declude Virus (http://www.declude.com)]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
|
