Re: [Declude.JunkMail] Declude bugs and problems with smartermail
Too funny! I thought I had heard more success than that with 2.0.6. I forgot that the 3.0 version for SmarterMail wasn't quite out yet. Can any other Declude/SmarterMail users comment on their success, and what they did to achieve it? It might be worth posting your config here in case someone who has been using SmarterMail can point out a configuration change that would help. I believe there's a THREADS parameter in the Declude.cfg in 2.0.6 that can be increased from the default to better handle the SPOOL/PROC directory under even moderate load. One of the main reasons we haven't tried the Declude/SmarterMail combo yet is SmarterMail's inability to give Declude SMTP AUTH info. Whitelisting authenticated users is pretty important with our setup, since we have the CMDSPACE and some dynamic IP tests weighted pretty heavily, and our customers have to have that setting in place anyway since we're not an ISP. Not having SMTP AUTH whitelisting would reduce our catch rate significantly since we'd have to lower those test weights quite a bit. Darin. - Original Message - From: Dave Beckstrom [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Sunday, September 04, 2005 12:55 AM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, It's amazing how much time servers and software can suck up. Pretty soon you're working almost around the clock I actually am running the 2.0.6 version of Declude for Smartermail. As long as I don't try to HOLD spam it seems that only about 2 out of every 10 spams makes it through with no Declude headers attached to the message. The software is pretty buggy. With HOLD turned off the .VIR directories are being cleaned up in the SPOOL/PROC directory now, too. It seems that the logic in the program changes with different settings in the config file and right now I have it set at the lesser of two evils. At least 80% of the spam is being processed by Declude now. The rest of the spam comes through to my in-box untouched. I'm starting to think that maybe my wife is going to dump me because my manhood isn't large enough and that I just can't live without a Rolex watch! LOL! -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Saturday, September 03, 2005 11:29 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Declude bugs and problems with smartermail Hi Dave, I know what you mean. After the first startup venture in the late 90's, holidays have never been the same... You might try running the 2.06 version for SmarterMail. I've heard mostly success for that. That may buy you some time until the kinks get worked out of the 3.0 beta. Darin. - Original Message - From: Dave Beckstrom [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Saturday, September 03, 2005 10:19 PM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, Ever since I started working out of a home office I do forget about the holidays! Well, the good news is that I've made some progress in understanding the problem. As long as I don't specify a HOLD action (EG. WEIGHT30 HOLD %DATE%) and instead run with (WEIGHT30 SUBJECT [SPAM]) then Declude will at least process MOST of the incoming email. When I tell Declude to hold email above a certain weight, it falls all over itself with problems. I'm seeing messages in the logs about attempts to move non-existent files. I see paths for these files having spaces or double slashes in the path names which are obvious programming errors. I'm half tempted to edit the Declude.exe file with a hex editor and fix the path problems myself. But there are some logic problems too so there is no point. I hate the thought of running for 3 days not being able to block ANY spam at all. But what are you going to do? At least I can tag the majority of the spam. -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Saturday, September 03, 2005 8:15 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Declude bugs and problems with smartermail Hi Dave, Probably not... Monday's Labor Day. Easy to forget those little things called holidays, isn't it? grin Darin. - Original Message - From: Dave Beckstrom [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Saturday, September 03, 2005 7:42 PM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Gary, Yeah, that sounds exactly like what is happening and I see messages in the log, as well, that supports what you're saying. Hopefully Declude support will be around on Monday and maybe we can work on getting that one solved. -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail-
RE: [Declude.JunkMail] Declude bugs and problems with smartermail
Darin, Here are a couple of things I found. A few minutes ago, I set log level to DEBUG so that I could show you the problems, and I changed my WIEGHT30 from a value of WEIGHT30 SUBJECT [SPAM] to WEIGHT30 HOLD %DATE% to enable holding of spam. Here is a log snippet with HOLD in effect: 09/04/2005 10:53:14.565 68454536903 [5476] Moving file to spam hold directory [D:\APPS\SMARTERMAIL\SPOOL\proc\68454536903.EML][D:\APPS\SMARTERMAIL\SPOOL\s pam\04 Sep 2005 \68454536903.EML] 09/04/2005 10:53:14 68454536903 ERROR: Could not move spam to hold! Code: [3] Error String: [The system cannot find the path specified.] Note the space after 04 Sep 2005 and before the slash. That is one bug in Declude. On a different topic, while we're talking about this, I'd like to suggest that Declude name the date folders something like 04_Sep_2005 with no spaces anywhere in the path. The reason is because people may wish to run DOS batch scripts against the folder. I'm pretty sure that DOS won't accept spaces in a directory path unless it's coded with double quotes in the path. EG (D:\APPS\SMARTERMAIL\SPOOL\spam\04 Sep 2005\68454536903.EML) Thus if Declude avoids spaces in the path to begin with someone who later wants to write a bat file to delete old spam folders will have a much easier time of it. Okay, back to the topic of bugs. Here is the next problem If I change the WEIGHT30 to WEIGHT30 HOLD (no date parameter) with the idea the spam would go directly into the SPOOL/SPAM directory, then another bug crops up. The path in the log would look something like this: [D:\APPS\SMARTERMAIL\SPOOL\proc\68454536903.EML][D:\APPS\SMARTERMAIL\SPOOL\s pam\\68454536903.EML] Note the double slashes before the spool file name. Declude didn't account for the fact that if you're not using their new date folders then one of those slashes in the path must be removed. Also, with HOLD in effect, over time messages will start to back up in the SPOOL/PROC directory and never be moved back into the SPOOL directory for re-processing. I have to manually copy them. Also, ironically, if I have HOLD in effect than the directories that appear in the SPOOL/PROC folder with a name such as 68454536903.VIR will never be cleaned up. They will exist until I delete them. If I do not have HOLD in effect, then Declude will remove those .VIR folders from the PROC directory when it's done processing them. So this is yet another bug I found. The third problem I'm fighting is a situation where Declude analyzes an email and may recognize it as spam but it still passes through Declude and arrives in my in-box with no Declude headers attached to the message. I can tell Declude processed it because the logs show it being analyzed, failing and weights being set. Yet the subject isn't modified and when I view the mail headers there is no sign of Declude ever having looked at the message. I think this may have some problem to do with copied or forwarded messages or perhaps some problem with email moving back and forth between the PROC and SPOOL folder and not being processed properly. I'll keep looking at this issue to see if I can find a pattern with the overlooked messages. Thanks, Dave -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Sunday, September 04, 2005 8:32 AM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Declude bugs and problems with smartermail Too funny! I thought I had heard more success than that with 2.0.6. I forgot that the 3.0 version for SmarterMail wasn't quite out yet. Can any other Declude/SmarterMail users comment on their success, and what they did to achieve it? It might be worth posting your config here in case someone who has been using SmarterMail can point out a configuration change that would help. I believe there's a THREADS parameter in the Declude.cfg in 2.0.6 that can be increased from the default to better handle the SPOOL/PROC directory under even moderate load. One of the main reasons we haven't tried the Declude/SmarterMail combo yet is SmarterMail's inability to give Declude SMTP AUTH info. Whitelisting authenticated users is pretty important with our setup, since we have the CMDSPACE and some dynamic IP tests weighted pretty heavily, and our customers have to have that setting in place anyway since we're not an ISP. Not having SMTP AUTH whitelisting would reduce our catch rate significantly since we'd have to lower those test weights quite a bit. Darin. - Original Message - From: Dave Beckstrom [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Sunday, September 04, 2005 12:55 AM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, It's amazing how much time servers and software can suck up. Pretty soon you're working almost around the clock I actually am running the 2.0.6 version of
RE: [Declude.JunkMail] Declude bugs and problems with smartermail - Folder Names
On the subject of folder names. Second the vote on no spacxes in file or folder names. Might it be better to use a simple format of 20050905 or 050905?? Also consider the issue of moving into a new year and the visual order of the folders when sorting by file name. This coming from using a GUI perspective when managing and troubleshooting date related issues. Mike -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dave Beckstrom Sent: Sunday, September 04, 2005 9:30 AM To: Declude.JunkMail@declude.com Cc: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, Here are a couple of things I found. A few minutes ago, I set log level to DEBUG so that I could show you the problems, and I changed my WIEGHT30 from a value of WEIGHT30 SUBJECT [SPAM] to WEIGHT30 HOLD %DATE% to enable holding of spam. Here is a log snippet with HOLD in effect: 09/04/2005 10:53:14.565 68454536903 [5476] Moving file to spam hold directory [D:\APPS\SMARTERMAIL\SPOOL\proc\68454536903.EML][D:\APPS\SMARTERMAIL\SPOOL\s pam\04 Sep 2005 \68454536903.EML] 09/04/2005 10:53:14 68454536903 ERROR: Could not move spam to hold! Code: [3] Error String: [The system cannot find the path specified.] Note the space after 04 Sep 2005 and before the slash. That is one bug in Declude. On a different topic, while we're talking about this, I'd like to suggest that Declude name the date folders something like 04_Sep_2005 with no spaces anywhere in the path. The reason is because people may wish to run DOS batch scripts against the folder. I'm pretty sure that DOS won't accept spaces in a directory path unless it's coded with double quotes in the path. EG (D:\APPS\SMARTERMAIL\SPOOL\spam\04 Sep 2005\68454536903.EML) Thus if Declude avoids spaces in the path to begin with someone who later wants to write a bat file to delete old spam folders will have a much easier time of it. Okay, back to the topic of bugs. Here is the next problem If I change the WEIGHT30 to WEIGHT30 HOLD (no date parameter) with the idea the spam would go directly into the SPOOL/SPAM directory, then another bug crops up. The path in the log would look something like this: [D:\APPS\SMARTERMAIL\SPOOL\proc\68454536903.EML][D:\APPS\SMARTERMAIL\SPOOL\s pam\\68454536903.EML] Note the double slashes before the spool file name. Declude didn't account for the fact that if you're not using their new date folders then one of those slashes in the path must be removed. Also, with HOLD in effect, over time messages will start to back up in the SPOOL/PROC directory and never be moved back into the SPOOL directory for re-processing. I have to manually copy them. Also, ironically, if I have HOLD in effect than the directories that appear in the SPOOL/PROC folder with a name such as 68454536903.VIR will never be cleaned up. They will exist until I delete them. If I do not have HOLD in effect, then Declude will remove those .VIR folders from the PROC directory when it's done processing them. So this is yet another bug I found. The third problem I'm fighting is a situation where Declude analyzes an email and may recognize it as spam but it still passes through Declude and arrives in my in-box with no Declude headers attached to the message. I can tell Declude processed it because the logs show it being analyzed, failing and weights being set. Yet the subject isn't modified and when I view the mail headers there is no sign of Declude ever having looked at the message. I think this may have some problem to do with copied or forwarded messages or perhaps some problem with email moving back and forth between the PROC and SPOOL folder and not being processed properly. I'll keep looking at this issue to see if I can find a pattern with the overlooked messages. Thanks, Dave -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Sunday, September 04, 2005 8:32 AM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Declude bugs and problems with smartermail Too funny! I thought I had heard more success than that with 2.0.6. I forgot that the 3.0 version for SmarterMail wasn't quite out yet. Can any other Declude/SmarterMail users comment on their success, and what they did to achieve it? It might be worth posting your config here in case someone who has been using SmarterMail can point out a configuration change that would help. I believe there's a THREADS parameter in the Declude.cfg in 2.0.6 that can be increased from the default to better handle the SPOOL/PROC directory under even moderate load. One of the main reasons we haven't tried the Declude/SmarterMail combo yet is SmarterMail's inability to give Declude SMTP AUTH info. Whitelisting authenticated users is pretty important with our setup, since we have the CMDSPACE and some dynamic IP tests weighted
Re: [Declude.JunkMail] Declude bugs and problems with smartermail
For us it takes about an hour a day of tweaks with very few complaints, and is well worth it for the additional revenue it brings in which is not to say that at some point in the future it might work better to outsource. We're positioned in the market as outsourcees rather than outsourcers, though. Darin. - Original Message - From: Robert E. Spivack [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Sunday, September 04, 2005 1:31 PM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Once in a while, it may be worthwhile to step back and count up all the time and figure out the real cost. We did that, and although we loved using Declude, it just wasn't the solution anymore. We outsourced to Postini, recovered the use of several hardware servers, and freed ourselves of at least 3 hours/day of babysitting declude and spam settings. Not suggesting it is a solution for everyone, but when we ran the numbers on not renewing software support contracts, reuse of hardware (saved buying a few more servers), and reallocation of our time (saved hiring another engineer), we actually are saving money and most importantly - no longer burned-out trying to fight spam. The end-user GUI is a big plus - problems of too little or too much spam getting through are now the responsibility of the end-user changing their settings and not us. For me, the straw that broke the camel's back was getting two emails from clients, within a few minutes of each other -- Your spam system sucks. It's letting too much junk through followed by Your spam system sucks. It's blocking important emails I need to receive. We'd tune and tune and tune and it work only work for a few days or a week. Then new spammers not covered by existing rbl's or filters would get through sending us back to square one. After two years of this, we finally decided we need to change our whole approach. And Postini has 24 x 7 telephone support - email is mission-critical; our clients still want email even on holidays :-) - sometimes there are things a big company can provide that you just can't get from a small one-- even a good one. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dave Beckstrom Sent: Saturday, September 03, 2005 9:56 PM To: Declude.JunkMail@declude.com Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, It's amazing how much time servers and software can suck up. Pretty soon you're working almost around the clock I actually am running the 2.0.6 version of Declude for Smartermail. As long as I don't try to HOLD spam it seems that only about 2 out of every 10 spams makes it through with no Declude headers attached to the message. The software is pretty buggy. With HOLD turned off the .VIR directories are being cleaned up in the SPOOL/PROC directory now, too. It seems that the logic in the program changes with different settings in the config file and right now I have it set at the lesser of two evils. At least 80% of the spam is being processed by Declude now. The rest of the spam comes through to my in-box untouched. I'm starting to think that maybe my wife is going to dump me because my manhood isn't large enough and that I just can't live without a Rolex watch! LOL! -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Saturday, September 03, 2005 11:29 PM To: Declude.JunkMail@declude.com Subject: Re: [Declude.JunkMail] Declude bugs and problems with smartermail Hi Dave, I know what you mean. After the first startup venture in the late 90's, holidays have never been the same... You might try running the 2.06 version for SmarterMail. I've heard mostly success for that. That may buy you some time until the kinks get worked out of the 3.0 beta. Darin. - Original Message - From: Dave Beckstrom [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Saturday, September 03, 2005 10:19 PM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, Ever since I started working out of a home office I do forget about the holidays! Well, the good news is that I've made some progress in understanding the problem. As long as I don't specify a HOLD action (EG. WEIGHT30 HOLD %DATE%) and instead run with (WEIGHT30 SUBJECT [SPAM]) then Declude will at least process MOST of the incoming email. When I tell Declude to hold email above a certain weight, it falls all over itself with problems. I'm seeing messages in the logs about attempts to move non-existent files. I see paths for these files having spaces or double slashes in the path names which are obvious programming errors. I'm half tempted to edit the Declude.exe file with a hex editor and fix the path problems myself. But there are some logic problems too so there is no point. I hate the thought of running
Re: [Declude.JunkMail] Declude bugs and problems with smartermail - Folder Names
Totally agree that MMDD works better for date-stamped directory names...especially if you're sorting. Good points about the bugs in not quoting directory names with spaces, or accounting for no date-stamped directories, but as a temporary workaround you could use the structure they intended, reporting the bugs and migrating back to your desired config after they're fixed. Have you tried increasing the number of threads to deal with the PROC folder backup? The messages not processed seems like the most unclear, but that could still be due to a particular config. I would suggest again posting the configs, if not to the list then at least to Declude support. Darin. - Original Message - From: Michael Jaworski [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Sunday, September 04, 2005 12:50 PM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail - Folder Names On the subject of folder names. Second the vote on no spacxes in file or folder names. Might it be better to use a simple format of 20050905 or 050905?? Also consider the issue of moving into a new year and the visual order of the folders when sorting by file name. This coming from using a GUI perspective when managing and troubleshooting date related issues. Mike -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Dave Beckstrom Sent: Sunday, September 04, 2005 9:30 AM To: Declude.JunkMail@declude.com Cc: [EMAIL PROTECTED] Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Darin, Here are a couple of things I found. A few minutes ago, I set log level to DEBUG so that I could show you the problems, and I changed my WIEGHT30 from a value of WEIGHT30 SUBJECT [SPAM] to WEIGHT30 HOLD %DATE% to enable holding of spam. Here is a log snippet with HOLD in effect: 09/04/2005 10:53:14.565 68454536903 [5476] Moving file to spam hold directory [D:\APPS\SMARTERMAIL\SPOOL\proc\68454536903.EML][D:\APPS\SMARTERMAIL\SPOOL\s pam\04 Sep 2005 \68454536903.EML] 09/04/2005 10:53:14 68454536903 ERROR: Could not move spam to hold! Code: [3] Error String: [The system cannot find the path specified.] Note the space after 04 Sep 2005 and before the slash. That is one bug in Declude. On a different topic, while we're talking about this, I'd like to suggest that Declude name the date folders something like 04_Sep_2005 with no spaces anywhere in the path. The reason is because people may wish to run DOS batch scripts against the folder. I'm pretty sure that DOS won't accept spaces in a directory path unless it's coded with double quotes in the path. EG (D:\APPS\SMARTERMAIL\SPOOL\spam\04 Sep 2005\68454536903.EML) Thus if Declude avoids spaces in the path to begin with someone who later wants to write a bat file to delete old spam folders will have a much easier time of it. Okay, back to the topic of bugs. Here is the next problem If I change the WEIGHT30 to WEIGHT30 HOLD (no date parameter) with the idea the spam would go directly into the SPOOL/SPAM directory, then another bug crops up. The path in the log would look something like this: [D:\APPS\SMARTERMAIL\SPOOL\proc\68454536903.EML][D:\APPS\SMARTERMAIL\SPOOL\s pam\\68454536903.EML] Note the double slashes before the spool file name. Declude didn't account for the fact that if you're not using their new date folders then one of those slashes in the path must be removed. Also, with HOLD in effect, over time messages will start to back up in the SPOOL/PROC directory and never be moved back into the SPOOL directory for re-processing. I have to manually copy them. Also, ironically, if I have HOLD in effect than the directories that appear in the SPOOL/PROC folder with a name such as 68454536903.VIR will never be cleaned up. They will exist until I delete them. If I do not have HOLD in effect, then Declude will remove those .VIR folders from the PROC directory when it's done processing them. So this is yet another bug I found. The third problem I'm fighting is a situation where Declude analyzes an email and may recognize it as spam but it still passes through Declude and arrives in my in-box with no Declude headers attached to the message. I can tell Declude processed it because the logs show it being analyzed, failing and weights being set. Yet the subject isn't modified and when I view the mail headers there is no sign of Declude ever having looked at the message. I think this may have some problem to do with copied or forwarded messages or perhaps some problem with email moving back and forth between the PROC and SPOOL folder and not being processed properly. I'll keep looking at this issue to see if I can find a pattern with the overlooked messages. Thanks, Dave -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Darin Cox Sent: Sunday, September 04, 2005 8:32 AM To: Declude.JunkMail@declude.com Subject: Re:
Re: [Declude.JunkMail] Declude bugs and problems with smartermail
Hi Dave, One comment. I don't think of Declude as an set it and forget it-type tool. It does take maintenance to stay on top of the spam problem... adding tests when needed, removing old tests, adjusting weights, etc... in general adjusting as the spam war changes. When we first installed it, we only did subject tagging and got about an 80% catch rate. For the first year we didn't do much to it except add a few new DNSBL or RHSBL tests here and there. It wasn't until a management change in 2003 that we had the freedom to pursue it more, switched to a hold policy with no subject tagging, and got our catch rate up around 99.5% with less than 0.5% false positives, much of it with the help of tools like Message Sniffer from sortmonster.com. If you haven't looked at it, I would highly recommend you do so. By itself, Sniffer has a catch rate around 95% on our systems. And while we do see almost daily false positives from Sniffer, part of it is due to a few customers that work in subjects very similar to typical spam... like a design/marketing firm that works with marriage and family issues, and has occasional discussions related to pornography... or a couple of electronic component distributors that do business with some Asian distributors that frequently spam. Pete is very good about maintaining a quality rulebase, though, and responds quickly to spam and false positive reports. I've heard good things about Invariant Systems invURIBL plug-in as well. Darin. - Original Message - From: Dave Beckstrom [EMAIL PROTECTED] To: Declude.JunkMail@declude.com Sent: Sunday, September 04, 2005 2:51 PM Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Robert, I hear you. I have to admit that I'm not all that happy with Declude at the moment. I dropped $900 on the software last week. Which is way too much money for this type of product when you consider all of the free options like the various postfix based solutions that are available. Ultimately, I decided to spend the money on Declude because the time I would invest in setting up 'nix based servers and learning these new solutions was time I don't have available. Why I'm not happy is because I've now invested several days in identifying all of these bugs. Moreover, to get them fixed will likely require yet more of my time working with support. So ultimately I'm probably looking at several more days of my time just to get a functioning product. The reason the problems didn't show up during testing is because I didn't want to activate the HOLD action until I was sure we didn't have a lot of false positives. I was short on time to test, too, because I wanted to take advantage of their sale price. I left the office for about two hours. Upon my return I now have 529 spams in my personal email in-box because I can't block them and I can only tag them. HeyI just had an idea!!! I bet I can specify a path on the HOLD action as a temporary workaround! Rather then using %DATE% or no path at all I bet if I specify a sub-directory it may work! I recall reading in the manual we can specify a path. Whoohoo! I have to leave right now for my father's birthday. I'll try my idea later this evening. That would rock if I could get hold working again! -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Robert E. Spivack Sent: Sunday, September 04, 2005 12:31 PM To: Declude.JunkMail@declude.com Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail Once in a while, it may be worthwhile to step back and count up all the time and figure out the real cost. We did that, and although we loved using Declude, it just wasn't the solution anymore. We outsourced to Postini, recovered the use of several hardware servers, and freed ourselves of at least 3 hours/day of babysitting declude and spam settings. Not suggesting it is a solution for everyone, but when we ran the numbers on not renewing software support contracts, reuse of hardware (saved buying a few more servers), and reallocation of our time (saved hiring another engineer), we actually are saving money and most importantly - no longer burned-out trying to fight spam. The end-user GUI is a big plus - problems of too little or too much spam getting through are now the responsibility of the end-user changing their settings and not us. For me, the straw that broke the camel's back was getting two emails from clients, within a few minutes of each other -- Your spam system sucks. It's letting too much junk through followed by Your spam system sucks. It's blocking important emails I need to receive. We'd tune and tune and tune and it work only work for a few days or a week. Then new spammers not covered by existing rbl's or filters would get through sending us back to square one. After two years of this, we finally decided we need to change our whole
RE: [Declude.JunkMail] Declude bugs and problems with smartermail
1) Declude leaves directories with names such as 6432144091.vir in the SPOOL/PROC overflow directory and it NEVER goes back and removes these directories or cleans them up. I have to manually delete the .vir directories. I haven´t seen that in my system smartermail/declude installation yet. I have seen it in my Imail/Declude installation. I manually delete those. I just don't bother. 2) Orphaned files. I have found files in the SPOOL/PROC directory where the .EML extension has been renamed to .EM$ and there is no matching .HDR file. These stay until I delete the orphans. I currently have a file in my SPOOL directory called X6432144091.EML with no matching .HDR file - which means its an orphan file too. 3) The PROC overflow directory is being populated with files even when the server is under very low utilization. This, in my opinion, portends major performance problems ahead of us when the server is under a high load. I don't see any tuning parameters available which allow me to tweak under what circumstances the PROC directory is utilized. I have a dual processor server with 2 gig of RAM on it and I should not be forced to the same limitations as someone with a single processor server. This process needs to be tunable. 4) This morning I had about 100 files in the SPOOL/PROC directory. I had to manually copy them to the SPOOL directory for processing. Once I moved them, the new files being placed into the PROC directory would automatically move back to the SPOOL directory for processing. So it appears that there is some situation where Declude forgets about some files in the SPOOL/PROC directory and never goes back and moves them back to the SPOOL. I can set up a script to do this (and delete the .vir folders too) every 15 minutes but I shouldn't have to do that. Before doing that check your logs to see what happened with those files. I only find a couple of emails once in that directory and looking at the declude logs and smartermails logs the files were delivered correctly. I didn't know why they stayed in the proc directory, but any way they were delivered fine. I just erased them. 5) This morning I had 45 spam emails in my in-box that had no header records indicating that they were ever processed by Declude. Apparently there is some situation where Declude doesn't process messages. I haven't yet figured out how or why this may be happening. I'll do more research to see what I can find. I am hoping that support will get with me ASAP and that together we can identify the cause of these problems and make some enhancements to Declude which will make it more smartemail friendly. That is strange. Reading your entire email, seems like there is a problem with your configuration files. Setting log debugging to high and working with support will help you find the problem. The file not found error you mentioned in another post seems to be the key to solve the strange declude behavior. Code:[3] Error String: [The system cannot find the path specified.]. That is strange, I had some errors like that when trying to manually indicate Declude config files where exactly to hold the spam I was receiving. I gave up and decided to try again later with that configuration. I just left it as the default configuration like this WEIGHT15HOLD %DATE% And it works without problems. It creates a date folder under my D:\SmarterMail\Spool\Spam\ directory and works just fine holding spam catched the corresponding day Declude support pretty sure will help you debug and solved the problem. I am confident that declude is working fine and doesn't have a bug as big as the one you point out... Or at least I hope so. It works for me. Regards Luis Arango __ [Email scanned for viruses] [Email escaneado contra virus] --- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type unsubscribe Declude.JunkMail. The archives can be found at http://www.mail-archive.com.
RE: [Declude.JunkMail] Declude bugs and problems with smartermail
Luis, Thank you so much for your post! When you said that you are running with HOLD %DATE% with no problems it got me to wondering. If it worked for you, why not for me? I thought about itthen a light went on... I opened up the $default$junkmail file and in my editor I turned on the visible spaces feature. I then entered HOLD %DATE% and I made sure there was no space or tab character after the %DATE% and voila! It appears to have worked. [6972] ACTION_HOLD - datahold [D:\APPS\SMARTERMAIL\SPOOL\spam\04 Sep 2005\68454540542.EML] [6972] ACTION_HOLD - reciphold [D:\APPS\SMARTERMAIL\SPOOL\spam\04 Sep 2005\68454540542.HDR] I then put a space after %DATE% in my $default$junkmail file and Declude broke again with the invalid path message in the log file. Who would have thought Declude would have used an invisible space after that parameter as part of the file path This is still a programming problem that Declude should attend to but at least we now know why it was failing on my end! Thanks for sharing about your experience. You gave me a clue that set me in a new direction that helped! Dave -Original Message- From: [EMAIL PROTECTED] [mailto:Declude.JunkMail- [EMAIL PROTECTED] On Behalf Of Panda Consulting S.A. Luis Alberto Arango Sent: Sunday, September 04, 2005 7:56 PM To: Declude.JunkMail@declude.com Subject: RE: [Declude.JunkMail] Declude bugs and problems with smartermail 1) Declude leaves directories with names such as 6432144091.vir in the SPOOL/PROC overflow directory and it NEVER goes back and removes these directories or cleans them up. I have to manually delete the .vir directories. I haven´t seen that in my system smartermail/declude installation yet. I have seen it in my Imail/Declude installation. I manually delete those. I just don't bother. 2) Orphaned files. I have found files in the SPOOL/PROC directory where the .EML extension has been renamed to .EM$ and there is no matching .HDR file. These stay until I delete the orphans. I currently have a file in my SPOOL directory called X6432144091.EML with no matching .HDR file - which means its an orphan file too. 3) The PROC overflow directory is being populated with files even when the server is under very low utilization. This, in my opinion, portends major performance problems ahead of us when the server is under a high load. I don't see any tuning parameters available which allow me to tweak under what circumstances the PROC directory is utilized. I have a dual processor server with 2 gig of RAM on it and I should not be forced to the same limitations as someone with a single processor server. This process needs to be tunable. 4) This morning I had about 100 files in the SPOOL/PROC directory. I had to manually copy them to the SPOOL directory for processing. Once I moved them, the new files being placed into the PROC directory would automatically move back to the SPOOL directory for processing. So it appears that there is some situation where Declude forgets about some files in the SPOOL/PROC directory and never goes back and moves them back to the SPOOL. I can set up a script to do this (and delete the .vir folders too) every 15 minutes but I shouldn't have to do that. Before doing that check your logs to see what happened with those files. I only find a couple of emails once in that directory and looking at the declude logs and smartermails logs the files were delivered correctly. I didn't know why they stayed in the proc directory, but any way they were delivered fine. I just erased them. 5) This morning I had 45 spam emails in my in-box that had no header records indicating that they were ever processed by Declude. Apparently there is some situation where Declude doesn't process messages. I haven't yet figured out how or why this may be happening. I'll do more research to see what I can find. I am hoping that support will get with me ASAP and that together we can identify the cause of these problems and make some enhancements to Declude which will make it more smartemail friendly. That is strange. Reading your entire email, seems like there is a problem with your configuration files. Setting log debugging to high and working with support will help you find the problem. The file not found error you mentioned in another post seems to be the key to solve the strange declude behavior. Code:[3] Error String: [The system cannot find the path specified.]. That is strange, I had some errors like that when trying to manually indicate Declude config files where exactly to hold the spam I was receiving. I gave up and decided to try again later with that configuration. I just left it as the default configuration like this WEIGHT15 HOLD %DATE% And it works without problems. It creates a date folder under my D:\SmarterMail\Spool\Spam\ directory and works just fine
