Re: [sniffer]Sniffer updates down?
Hi John, I got my Sniffer update at 5:03 pm no problem from Toronto Goran Jovanovic Omega Network Solutions -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of John T (Lists) Sent: Friday, June 02, 2006 5:23 PM To: Message Sniffer Community Subject: [sniffer]Sniffer updates down? I am getting errors since late last night that host can not be found. John T eServices For You Seek, and ye shall find! # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
[sniffer]Ebay Phishing Emails getting through
Hi, I normally see maybe 6 to 10 phishing e-mails per day for the volume of mail that I handle (~15,000 msg/day). Yesterday was an explosion in my terms. HTML.PHISHING.BANK.GEN088.SANESECURITY.0603080..52 HTML.PHISHING.BANK.GEN615.SANESECURITY.06051202.F6 HTML.PHISHING.BANK.GEN220.SANESECURITY.0603240...4 HTML.PHISHING.CARD.SANESECURITY.0602210..4 HTML.PHISHING.BANK.GEN015.SANESECURITY.0602180...1 HTML.PHISHING.BANK.GEN055.SANESECURITY.0603050...1 I catch these and treat them as a virus using CLAM AV and the SANE Security database. Goran Jovanovic Omega Network Solutions Tel: 416 322-0333 -Original Message- From: Message Sniffer Community [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Thursday, May 18, 2006 10:33 AM To: Message Sniffer Community Subject: [sniffer]Re[2]: [sniffer]Ebay Phishing Emails getting through Hello Andrew, Wednesday, May 17, 2006, 5:35:36 PM, you wrote: Certainly, submitting samples to spam@ (or preferably your local spam submission point polled by our bots) will put these messages in front of us if we have not already created rules for them. I've just manually submitted the ~35 messages that my filters triggered on for phishing that didn't trigger Message Sniffer today but ended up in my HOLD folder anyway due to their total spamminess. Most of them are against eBay and came from Germany. If your overall false positive rate is low enough then it would be great if you could automate that process to create a synthetic spamtrap. Somehow, take the most spammy of the messages that get past SNF and send them to a special account on your system from which our robots could pull the messages Since we code rules 24x7x365 we would be able to respond to these quickly and (from your perspective) automatically. _M -- Pete McNeil Chief Scientist, Arm Research Labs, LLC. # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED] # This message is sent to you because you are subscribed to the mailing list sniffer@sortmonster.com. To unsubscribe, E-mail to: [EMAIL PROTECTED] To switch to the DIGEST mode, E-mail to [EMAIL PROTECTED] To switch to the INDEX mode, E-mail to [EMAIL PROTECTED] Send administrative queries to [EMAIL PROTECTED]
RE: Re[4]: [sniffer] When to go persistent
Hi, I just got my service up and running using Matt's post http://www.mail-archive.com/sniffer@sortmonster.com/msg00169.html It was simple especially since I already the resource kit installed. Now I know that this I supposed to work to get the persistent instance to load the new rulebase after a download. REM Load new rulebase file. %LicenseID%.exe reload But is there any way to query the service and ask it to tell you when was the last time the rulebase was loaded? Or what version of the rulebase it is using? When running in peer mode this question does not arise since the instances read the file off disk so there is no problem. With the persistent instance this is not the case and I would like to know that it really is using the newest rulebase. Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Thursday, February 23, 2006 3:11 PM To: Rick Robeson Subject: Re[4]: [sniffer] When to go persistent On Thursday, February 23, 2006, 1:22:53 PM, Rick wrote: RR I thought you had to run this as a service? RR Rick Robeson RR getlocalnews.com RR [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] Strictly speaking you do not have to run it as a service, but it is more convenient to do so. If you run it from the command line then you would need to remain logged in. Running the persistent instance from the command line is convenient for testing, but it is much better to run it as a service in a production environment - that way it starts and stops with the other services as expected, doesn't require any account to be logged in, etc... _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] When to go persistent
Hi, Is there any good rule of thumb, in terms of messages processed per minute/hour/day when you should move to a persistent instance of Sniffer? Thank you Goran Jovanovic Omega Network Solutions This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] When to go persistent
Andrew, So when you went to persistent it lowered the stress on your already stressed hardware? And I see that Pete has responded as I write this with: Use it Well I will set it up and see how my system reacts. Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Colbeck, Andrew Sent: Thursday, February 23, 2006 11:39 AM To: sniffer@SortMonster.com Subject: RE: [sniffer] When to go persistent Goran, I'd be interested in Pete's technical answer, too. The practical answer is that you should always go with the persistent instance of Message Sniffer. From reading Pete's previous screeds and monitoring the list here in the last year and from having my own troubles, it's pretty clear to me that only marginal cases suffer with the persistent mode (and I was one of them). Pete's answer on volumes won't answer what are the marginal cases, it just doesn't fit your question. For me, it was simple lack of hardware, but I was *right* on the edge. Andrew 8) -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Goran Jovanovic Sent: Thursday, February 23, 2006 8:30 AM To: sniffer@SortMonster.com Subject: [sniffer] When to go persistent Hi, Is there any good rule of thumb, in terms of messages processed per minute/hour/day when you should move to a persistent instance of Sniffer? Thank you Goran Jovanovic Omega Network Solutions This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] When to go persistent
Pete, To run in persistent mode, simply launch an instance of SNF from the command line with the word persistent in place of the file to scan. licenseid.exe authentication persistent I am calling Sniffer from Declude. Could I just later my statement in my config file to include persistent? That way the first time it is called that instance will go persistent and all the rest will end up talking to it? Regardless of how the persistent instance is started should I have the persistent keyword on the line that is called from Declude? Goran Jovanovic This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] What is this file
Thank you that is great. Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Thursday, February 23, 2006 3:08 PM To: Goran Jovanovic Subject: Re: [sniffer] What is this file On Thursday, February 23, 2006, 1:07:07 PM, Goran wrote: GJ Pete, GJ I have seen a couple of times that the file GJ C:\External\Sniffer\my license-20060221071316x386D4931-2352.SVR GJ Is open and cannot be backed up. GJ What is this file? I assume that I do not need to be worried since the GJ file disappears. When in peer-server mode, if an instance comes to life and finds it is the only instance around it will set itself up as a server just in case another instance comes along and needs help. When an instance of SNF is acting as a server it will announce that by creating a .SVR file in the working directory. In peer-server mode, a server-peer will handle a few jobs, then it's own, and then it will go away so it can return it's result. While it is active it will leave it's .SVR file out to advertise to the peer-clients that it is available to process messages. In persistent mode, the server-peer never has a message of it's own to process and so it never goes away (almost). As a result, all peer-clients always hand off their messages to the persistent peer-server. Since the persistent peer-server never goes away the .SVR file will also not go away. These files are all generally transient. (.QUE, .FIN, .ABT, .XXX, etc...) This causes some trouble with backup software. It's usually best to skip backing up the sniffer working directory except for the .exe, .snf, and any script files you have. It is usually best to keep a current / recent copy of those files in a separate directory that can be backed up and to otherwise treat the SNF working directory as you would a temp directory. (skip it) Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Bad Rule - 828931
I just ran the grep command on my log and I got 850 hits. Now is there a way to take the output of the grep command and use it pull out the total weight of corresponding message from the declude log file, or maybe the subject? Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Sullivan Sent: Tuesday, February 07, 2006 7:47 PM To: Landry, William (MED US) Subject: Re[4]: [sniffer] Bad Rule - 828931 Hello William, Tuesday, February 7, 2006, 7:39:05 PM, you wrote: LWMU grep -c Final.*828931 c:\imail\declude\sniffer\logfile.log That's what I tried. Just figured out I forgot to capitalize the F. It works. Confirmed - 22,055 I'm writing a program now to parse the sniffer log file, extract the file ID, lookup the id in sql server, determine quarantine location, extract q/d pair from quarantine and send to user. -- Best regards, Davidmailto:[EMAIL PROTECTED] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[4]: [sniffer] Bad Rule - 828931
OK to answer my own question. Run the following commands grep -U Final.828931 snf.log 1.txt cut -b26-41 1.txt 2.txt grep -U -f2.txt d:\spool\dec0207.log 3.txt egrep -U \smd Tests failed|\smd Subject 3.txt 4.txt notepad 4.txt Now I have to read my 4.txt and figure out what I am going to do about it. Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Goran Jovanovic Sent: Tuesday, February 07, 2006 8:39 PM To: sniffer@SortMonster.com Subject: RE: Re[4]: [sniffer] Bad Rule - 828931 I just ran the grep command on my log and I got 850 hits. Now is there a way to take the output of the grep command and use it pull out the total weight of corresponding message from the declude log file, or maybe the subject? Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of David Sullivan Sent: Tuesday, February 07, 2006 7:47 PM To: Landry, William (MED US) Subject: Re[4]: [sniffer] Bad Rule - 828931 Hello William, Tuesday, February 7, 2006, 7:39:05 PM, you wrote: LWMU grep -c Final.*828931 c:\imail\declude\sniffer\logfile.log That's what I tried. Just figured out I forgot to capitalize the F. It works. Confirmed - 22,055 I'm writing a program now to parse the sniffer log file, extract the file ID, lookup the id in sql server, determine quarantine location, extract q/d pair from quarantine and send to user. -- Best regards, Davidmailto:[EMAIL PROTECTED] This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Stock SPAM now HTML
Will it ever stop :( Probably not. Actually maybe I shouldn't be wishing that SPAM stops because then I would lose a revenue streamhmm conundrum Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Thursday, February 02, 2006 7:20 AM To: Goran Jovanovic Subject: Re: [sniffer] Stock SPAM now HTML On Wednesday, February 1, 2006, 11:30:49 PM, Goran wrote: GJ GJ GJ GJ Well the plain text stock spam has just taken a turn to more GJ interesting and SNF is not capturing it yet as of 10:55 EST. I have submitted a couple to spam@ GJ GJ Now they are including part of a picture to make up the text. GJ Here is what the source looks like Isn't it amazing. I've coded some abstracts for this. More to come. _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Stock SPAM now HTML
This is going to get harder and harder to identify and fight. Is it worthwhile to put something like this in a new category which we are very confident about and so if it fails on the new combined image/text thing we can delete it outright? Not sure if this is a good idea or not but I had to add extra static filters to pop the older text only stock spam above my delete weight. This combined image/text is going to make it tougher I think. Thoughts? Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Thursday, February 02, 2006 11:40 AM To: Goran Jovanovic Subject: Re[2]: [sniffer] Stock SPAM now HTML There are some new mutations of the latest campaigns out today. These ones look like they were hand tweaked (not evolved by machine). They are a lot tougher, but I think we've got some abstracts coming out that will get them. This new trend - using embedded images, adding static to images to avoid hashing systems, stuffing text, and avoiding links and email addresses is going to increase. _M On Thursday, February 2, 2006, 11:12:59 AM, Goran wrote: GJ Will it ever stop :( GJ Probably not. Actually maybe I shouldn't be wishing that SPAM stops GJ because then I would lose a revenue streamhmm conundrum GJ Goran Jovanovic GJ Omega Network Solutions GJ -Original Message- From: [EMAIL PROTECTED] GJ [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Thursday, February 02, 2006 7:20 AM To: Goran Jovanovic Subject: Re: [sniffer] Stock SPAM now HTML On Wednesday, February 1, 2006, 11:30:49 PM, Goran wrote: GJ GJ GJ GJ Well the plain text stock spam has just taken a turn to more GJ interesting and SNF is not capturing it yet as of 10:55 EST. I GJ have submitted a couple to spam@ GJ GJ Now they are including part of a picture to make up the text. GJ Here is what the source looks like Isn't it amazing. I've coded some abstracts for this. More to come. _M This E-Mail came from the Message Sniffer mailing list. For GJ information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html GJ This E-Mail came from the Message Sniffer mailing list. For GJ information and (un)subscription instructions go to GJ http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Stock SPAM now HTML
Well the plain text stock spam has just taken a turn to more interesting and SNF is not capturing it yet as of 10:55 EST. I have submitted a couple to spam@ Now they are including part of a picture to make up the text. Here is what the source looks like CHINA WORLimg src="" CORP. br Syimg src="" br Price $img src="" br Shares out: img src="" Million br Market Capitimg src="" Million br Significant Revenue Growth iimg src="" br Averagimg src="" br Rating: Stroimg src="" Buy br 7 days trading img src="" $2.50 br 30 day trading target: $3.img src="" br Goran Jovanovic Omega Network Solutions
[sniffer] The SPAM bots?
Hi, Are the bots working again? I am seeing a number of the STOCK pitches coming through (the ones that use the picture attachment eg. tdimg border=0 alt= src=cid:a8c0936faa69131141800cf3347d17a4/td) Sniffer did not catch the message and I have forwarded it to SPAM@ Thanx Goran Jovanovic Omega Network Solutions This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] The SPAM bots?
Thanks Pete, I think I am seeing a slowdown of this type of SPAM getting through now. Goran Jovanovic Omega Network Solutions -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Monday, January 30, 2006 7:20 PM To: Goran Jovanovic Subject: Re: [sniffer] The SPAM bots? On Monday, January 30, 2006, 10:16:06 AM, Goran wrote: GJ Hi, GJ Are the bots working again? I am seeing a number of the STOCK pitches GJ coming through (the ones that use the picture attachment eg. GJ tdimg border=0 alt= GJ src=cid:a8c0936faa69131141800cf3347d17a4/td) GJ Sniffer did not catch the message and I have forwarded it to SPAM@ There was a lot of that today. No, the bots are off until further notice. I think we have the image spam under control for the moment. Thanks, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
[sniffer] Hit Rate Discrepancy
Hi, I think I am having a problem with my Declude log file numbers/stats and I want to try and figure it out. Last week my Sniffer hit rate went from SNIFFER6,699...64.78% To yesterday SNIFFER1,299...10.24% This is wrong as Sniffer should and does trigger much more often (more like the first one) So I looked in the Sniffer log from yesterday and tried to do some quick stats. There are: Final 8573 67.6% Clean 4104 32.4% Total 12677 And the total compares to the total number of messages processed Total Messages Processed: 12,685 So am I interpreting the Sniffer log correctly? Do I need to worry about the Match entries? Thanx Goran Jovanovic The LAN Shoppe This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] RAID Levels for Spool Folder
Matt and Charles, Thank you for your insight and comments. Now I just have to go and get the money to get something that I want :) Goran Jovanovic The LAN Shoppe This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] Moving Sniffer to Declude/SmarterMail
John, It is a well known and published fact (on the Imail list) that RAID5 should never ever be used for the spool directory or any other directory that has a high write activity. This is basic physics. RAID5 should really only be used for high read activity only, such as databases where most of the writing is done to transaction (log) files and at spaced intervals those transactions are committed to the database. RAID1 or even RAID0+1 is best for the spool and logs. I guess this is going against what I think should be happening. In a RAID 5 array the write to the drives is broken into many smaller writes along with the data protection/CRC info and then those writes are written to different drives. It seems to me that it should be faster to do a bunch of small writes rather than 1 big write. What am I missing? Goran Jovanovic The LAN Shoppe This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: Re[2]: [sniffer] Moving Sniffer to Declude/SmarterMail
OK that is for hardware level RAID. I had thought that you would offset the extra processing time by being able to write less to each drive. Now does anyone know how much overhead Windows 2000/2003 software RAID 1 on dynamic disks produces over hardware level RAID 1? I am assuming it would be substantial. Goran Jovanovic The LAN Shoppe -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Pete McNeil Sent: Wednesday, March 16, 2005 11:43 AM To: Goran Jovanovic Subject: Re[2]: [sniffer] Moving Sniffer to Declude/SmarterMail On Wednesday, March 16, 2005, 11:25:46 AM, Goran wrote: snip/ GJ I guess this is going against what I think should be happening. In a GJ RAID 5 array the write to the drives is broken into many smaller writes GJ along with the data protection/CRC info and then those writes are GJ written to different drives. It seems to me that it should be faster to GJ do a bunch of small writes rather than 1 big write. GJ What am I missing? Writing data to a single hard drive takes x amount of work. Writing data to more than one drive takes x+y amount of work where y is breaking up the data into chunks. Writing data to a raid 5 takes x+y+z amount of work where y is described above and z is calculating a CRC stripe which must now also be saved to a hard drive. So, writing to raid5 is relatively very expensive compared to writing to a plain old hard drive, or a less complex raid (such as mirroring). IMO, the best strategy for email servers is to use an ordinary, single fast HD for all spool operations, and place mailboxes on a raid 1 or raid 10. Hope this helps, _M This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html This E-Mail came from the Message Sniffer mailing list. For information and (un)subscription instructions go to http://www.sortmonster.com/MessageSniffer/Help/Help.html
RE: [sniffer] RAID Levels for Spool Folder
Matt, I think that you sort of answered the question that I did not really ask. I was really trying to get information on the different performance levels for of S/W vs H/W RAID for an ideal scanning only box. So let me try this out and people can comment All SCSI 15K drives with HW RAID controller 2 x 36 GB drives R1 on first channel (36 GB usable) C Windows 10 GB D IMAIL/Smartermail/Declude files/Declude filters per domain configs/banned files (5 days only) 20 GB P Page volume 3 GB 3 x 36 GB drives R5 on second channel (72 GB usable) L Logs for JM, Virus, IMAIL/SmarterMail, Sniffer, invURIBL, et al 10 GB S Storage for all daily logs 60 GB 1 x 36 GB Hot Spare drive From what we have discussed here drive L will get hit a lot. If you create a process that Matt is describing to move the active logs from L to S you should not worry about running out of space on the L drive. Now looking back I am not sure if I have crafted this well since the SPOOL files for IMAIL will end up on D. Is there a way to move them for Smartermail as there does not seem to be a way to move them in IMail? The good part of this config is that the spool files which have a lot of read/write are on a different volume/channel from the other log files. I am not sure what amount of space you should allocate to a server that would process 100,000+ messages a day? Anyone have comments on this config. Thanx Goran Jovanovic The LAN Shoppe From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Wednesday, March 16, 2005 3:49 PM To: sniffer@SortMonster.com Subject: Re: [sniffer] RAID Levels for Spool Folder IMO, Software RAID is not the way to go on a busy machine. You will save a measurable amount of overhead by going with hardware based RAID of any sort since the controller should handle the processes associated with the RAID. Note that this isn't the case with inexpensive RAID controllers such as the cheaper IDE and SATA controllers which still place a fair burden on the OS/processor. True RAID cards also offer additional cache which can speed up the performance on reads, and also on writes if you are battery backed up (otherwise don't use write caching because you could lose or corrupt data during a power outage). There's also several common misconceptions about what is proper to do for a mail server. RAID 5 is the best choice under almost all conditions. The trick here is that while RAID 10 offers both redundancy in mirroring and speed in striping, most servers have a limited amount of space for disks. So a server with 6 disks will operate with the speed of 3 disks spanned in a RAID 10 configuration, but 6 disks in RAID 5 will operate as 5 disks spanned plus a little bit of overhead, though not nearly enough so that it falls short of the performance of just 3 disks in a simple span. Therefore RAID 5 should be the default choice for speed in such an environment. Another misconception is that data is always striped in RAID 0 or RAID 5. This depends on the file size and the stripe size. Most stripes are 64 KB (configurable in most setups). If you have some form of striping for your spool drive, most messages fall far under 64 KB and will only get written to one disk (CRC will also get written in RAID 5). Therefore for a spool folder, RAID 5 with 3 drives (the minimum), will perform rather closely to RAID 5 with 10 drives since most files will only land on one disk (with the other corresponding stripes containing no data). The MFT however for a drive with a lot of files will grow to be quite large and benefits from having multiple disks, and opening very large files such as logs will also benefit from having many disks. There is also an advantage to seek times when having multiple disks, especially if you keep your partitions sized small for performance. I've run a dual processor 3.06 Ghz server with both 6 Seagate 15,000 RPM drives in RAID 5 and the same with 3 Seagate 10,000 RPM drives in RAID 5 running on a less capable controller, and there was no impact on performance while the server was handling over 125,000 unique messages a day. The only noticeable difference was the time it would take to open a 500 MB log file, or the time it would take to enumerate the file names from the MFT on a partition that contained tens of thousands of files in the root. It seems quite apparent that with modern processors, even in dual processor configurations, that you will run out of CUP cycles long before you run out of disk I/O in a well managed RAID 5, 3 drive configuration on an IMail/Declude/Sniffer server. Take note that the log files for Declude, Sniffer and IMail all become massively fragmented, and if you don't have a process to remove these from active partitions on your server or defragment them, then performance will be severely impacted. I run a job hourly that copies all such logs to a different partition and combines them with older
