Will, I know someone else also suggested this, but I wanted to make sure I echoed it. Make sure you go through this post if you are running 2.0.6 - http://www.mail-archive.com/[email protected]/msg24658.html
I had that exact same issue - my overflow would fill up even though the box did not appear to be taxed. Add the DNSOVERRIDE in the declude.cfg file it made a huge difference - fixed my issue. Darrell ------------------------------------------------------------------------ Check out http://www.invariantsystems.com for utilities for Declude And Imail. IMail/Declude Overflow Queue Monitoring, SURBL/URI integration, MRTG Integration, and Log Parsers. Darin Cox writes:
MessageHmmm...doesn't look like a DNS timeout issue, then, and the entire message processed in under 2/3 of a second.. The relevant lines seem to be: 08/01/2005 10:45:49.796 Q358E016100003310 [5904] Processes - CURRENT [15] - MAX ALLOWED [25] 08/01/2005 10:45:49.796 Q358E016100003310 [5904] Under max, trying overflow - PASSING [E:\IMAIL\SPOOL\overflow] 08/01/2005 10:45:49.796 Q358E016100003310 [5904] Checking overflow folder [E:\IMAIL\SPOOL\overflow] Are you using Declude 2.06? It looks like you may be from the logs. If so, there is a MAXPROCESSES parameter for the Declude.CFG file to increase the number of processes available for scanning of messages, but I would contact Declude support to see why this went to overflow since the second line above seems to indicate it should process since there are only 15 running out of 25 processes available....plus the fact that the message had already been processed and just needed to be handed back to IMail for delivery. I may be reading this wrong, though. We're still on 1.82. Darin. ----- Original Message ----- From: Will To: [email protected] Sent: Tuesday, August 02, 2005 2:31 PM Subject: RE: [Declude.JunkMail] Declude Woes Final Update: My mail server is running perfectly smooth processing mail as soon as it arrives while running Declude Antivirus and three DNS blacklists using Imails built-in antispam. I have tried to re-enabled Declude junkmail, but it immediately starts to backlog the spool directory again and mail processing is slow. Perhaps someone can see something that I do not in this log file. This is a sample connection with debug logging on. I have a service contract with Declude and Imail for what that's worth, but I'm not going to get anywhere with that unless I can help point them in the right direction so they deem me worthy to forward my issue onto someone with experience. I'm currently processing what appears to be 100,000 messages per day with 3000 accounts (90% of that incoming mail). My previous counts of 200,000 only appear when declude junkmail is running, so they are not accurate. Will -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Will Sent: Monday, August 01, 2005 9:05 AM To: [email protected] Subject: RE: [Declude.JunkMail] Declude Woes Just an update. Last week I had disabled Declude and cleaned out my spool directory. For the next day I had not experienced any issues. My spooled messages are staying well under 500 and mail is flowing exactly how I want it to (without Declude). Now, we can get away with not having Declude Junkmail for a bit, but I really needed virus scanning, so I re-enabled Declude Antivirus. After I verified everything was running smoothly I downloaded F-prot and removed Mcafee before the weekend and things have also been running very smooth. Just a note, I had run Declude Antivirus for years without any issues and I only seem to run into these overflow problems when I introduce Declude Junkmail. Since the major issues I had last week I have made three changes. I have modified the queue manager to process 100 messages at a time. I have disabled Declude Junkmail. I have now moved from Mcafee to F-Prot. The only one of these changes that has ensured that mail does not go into overflow and backup is the disabling of Declude Junkmail. I'm not so sure it's a DNS issue because the Imail spam filters run perfectly fine, which I am now using in place of Declude. They do not do as good a job identifying spam, but they are better than nothing. Things to think about: I was interested in the statement that the mail server can be setup to not bounce messages if they are detected as SPAM. This still confuses me, because the BOUNCEONLYIFYOUMUST statement in the $default$.junkmail file appears to actually bounce the messages. As it is, all my filters are set to WARN and I don't understand why changing them to BOUNCEONLYIFYOUMUST will help. I do not believe I have bulk mailers on my system, but I will not discount it. Does anyone have a good suggestion for determining this? After looking through my logs and running Imail log analyzer none of my own IP's stand out for message counts and the invalid user entries I find are from outside sources outside of my network. These connections are always denied and mail does not get processed onto our server. I will think about Declude HiJack. I still want to use Declude Junkmail, so I will test some of the suggestions put into place such as whitelisting my IP range and disabling outbout virus scanning, but other than these two suggestions, I don't see any major changes here that will affect me. Will Friday: SpamContent 13400 SpamPhrase 19107 SpamFeatures 1283 SpamUrlDomain 35018 LocalDeliver 92331 RemoteDeliver 10526 Saturday: SpamContent 10089 SpamPhrase 20739 SpamFeatures 1090 SpamUrlDomain 27194 LocalDeliver 83297 RemoteDeliver 9008 Sunday: SpamContent 10425 SpamPhrase 18741 SpamFeatures 793 SpamUrlDomain 25756 LocalDeliver 81033 RemoteDeliver 10142 -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Matt Sent: Friday, July 29, 2005 2:03 PM To: [email protected] Subject: Re: [Declude.JunkMail] Declude Woes Will, What you probably have are users bulk-mailing through your server (evidenced by the GSE files), issues with running both IMail Antispam and Declude, and choosing a much slower than optimal virus scanner as your one an only engine in a CPU-challenged environment. There could well be issues with fragmentation and not enough disks as well. The reports that you shared are somewhat inconclusive, primarily because running Declude plus IMail Antispam will skew these numbers. So there is no telling what the true volume is. If you run Declude, you should turn IMail Antispam off and just rely on Declude alone. Your config looks fine except that you should strongly consider spending about $50 on an F-Prot license and using that instead of McAfee since you are CPU challenged. The fact that you are an ISP also speaks volumes to the need of restricting bulk-mailing among your client base. Not only can they redline your server, they can also get you blacklisted. Adding a gateway will do nothing for this part of the issue. You need to look into integrating Declude Hijack, but be prepared to sort through issues in the week that follows integrating it. To save CPU, you could whitelist your userbase in Declude by their IP space by listing them in your global.cfg. You could also turn off outbound virus scanning with Declude Virus Pro (not sure if that also applies to Declude Standard). IMO, this isn't an issue of sizing or a bug in Declude, this is all about optimizing your configuration where it counts most, and putting in place reasonable protections so that bulk-mailing isn't allowed to go through your mail server. Jumping to a gateway will just be another partial fix and it might not provide enough juice to prevent you from backing up, especially if the load is primarily the result of customers bulk-mailing through your server. If you did everything that I identified here, you would save about 1/2 to 2/3 of your current CPU utilization. That might not cut it however since you were clearly demanding more than 100% before, but I believe it should. Matt Will wrote: Well thank you to everyone who responded to my need for perspective and assistance. I believe I will be in need of a mail gateway if I am to continue to use Declude. Also, does anyone know how to configure an Imail/Declude setup to not bounce spam messages? Will -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Michael Jaworski Sent: Friday, July 29, 2005 1:11 PM To: [email protected] Subject: RE: [Declude.JunkMail] Declude Woes I can second the need for a gateway defense when under attack. We run a Windows shop and were being crushed under multiple dictionary attacks for two domains on a daily basis. I took the daunting task to build our first Linux box running Postfix. The first box was a tough start though I had a employee who had Linux experience. We are running Postfix on OpenBSD 3.6 with MySql for dynamic update ability. (I am still working on grabbing additions, updates and deletions from SmarterMail admins so we can throw all our domains in Postfix and update in realtime) After a few weeks we added a second box in the event the first box went down. The second box was a breeze since it was basically a duplication. Both mx records now point at the two boxes. The hardware was old 500Mhz and 1ghz cpu with 512mbs of ram each. The 1ghz is primary and takes 75% of the load without much effort with plenty of free memory. The whole setup allows the main server running SmarterMail/Declude Pro/Sniffer/F-Prot to respond quickly to POP, web mail and smtp traffic requests. The Linus approach only should cost you some time and old equipment as the software is free. Our experience over the last two years showed it was worth climbing the short Linux learning cliff. And it is true ... they run forever One important note not related to using a gateway: We never bounce spam e-mail back to the "sender". The backscatter traffic can kill you and skew your reports. Michael Jaworski Puget Sound Network, Inc. (206) 217-0400 (800) 599-9485 ------------------------------------------------------------------------------ From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Will Sent: Friday, July 29, 2005 9:38 AM To: [email protected] Subject: RE: [Declude.JunkMail] Declude Woes Well, I'm back at it today. Yesterday I disabled Declude early in the day and started working mail back into the spool directory from the overflow directory. This was a long process, but by the end of the day I had gone from a backlog of 150,000 files in the spool and 134,000 in the overflow directory to about 1500 (that includes logs). During this time I needed to stop and restart the queue manger a number of time. I did this to allow me to delete all the .gse files, which I figured would save me time discarding them. However, by the time I got down to 1500 files and started to watch the spool it started to increase in size again; climbing to 4000 within a matter of minutes. I stopped and restarted the queuemanager and these files were then processed. I verified they were actually getting processed by sending test messages to myself. At this point I was pretty upset and confused. I looked through the sys logs and found nothing out of the ordinary, queuemanger would simply stop. I set all the queuemanager setting back to default and tried again without luck. I had to stop and restart it every few minutes to get it to process a few thousand messages. Finally, I purchased an Imail service agreement and upgraded to 8.21. Magically, it worked. The queumanger started to deliver messages as soon as they arrived. My thought immediately went into conspiracy mode. It seems like this has happened before where we had a perfectly workable solution and something completely confusing happened and an upgrade magically fixes it! Anyway. I re-enabled declude and let it run overnight. Now I have a backlog again. There are mostly D*.SMD files in the spool right now with all their delivery Q* files in the overflow directory (*shakes fist at overflow directory*). Time to start the process again today. I'm disabling declude to get those messages out and one thing to note, after I have stopped the smtp server and added smtpd.exe backing into the delivery application, I still have about 20+ declude.exe processes. I have stopped and started it again as well as the queuemanager and they are still there. In fact they are creating more declude.exe processes as I watch. I'm trying to kill them, but they just keep coming back. having to restart so I can start processing mail. We are an ISP and here are some random examples of some of our Imail daily reports to give you an idea of what kind of traffic we see: -- =====================================================MailPure custom filters for Declude JunkMail Pro.http://www.mailpure.com/software/=====================================================
--- This E-mail came from the Declude.JunkMail mailing list. To unsubscribe, just send an E-mail to [EMAIL PROTECTED], and type "unsubscribe Declude.JunkMail". The archives can be found at http://www.mail-archive.com.
