-----Original Message-----
From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]On Behalf Of Will
Sent: Tuesday, August 02, 2005
11:31 AM
To: [email protected]
Subject: RE: [Declude.JunkMail]
Declude Woes
Final Update:
My mail server is running
perfectly smooth processing mail as soon as it arrives while running Declude
Antivirus and three DNS blacklists using Imails built-in antispam. I have
tried to re-enabled Declude junkmail, but it immediately starts to backlog the
spool directory again and mail processing is slow. Perhaps someone can
see something that I do not in this log file… This is a sample
connection with debug logging on.
I have a service contract
with Declude and Imail for what that’s worth, but I’m not going to
get anywhere with that unless I can help point them in the right direction so
they deem me worthy to forward my issue onto someone with experience.
I’m currently
processing what appears to be 100,000 messages per day with 3000 accounts (90%
of that incoming mail). My previous counts of 200,000 only appear when
declude junkmail is running, so they are not accurate.
Will
-----Original
Message-----
From:
[EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Will
Sent: Monday, August 01, 2005 9:05
AM
To: [email protected]
Subject: RE: [Declude.JunkMail]
Declude Woes
Just an
update…
Last
week I had disabled Declude and cleaned out my spool directory. For the
next day I had not experienced any issues. My spooled messages are
staying well under 500 and mail is flowing exactly how I want it to (without
Declude). Now, we can get away with not having Declude Junkmail for a
bit, but I really needed virus scanning, so I re-enabled Declude
Antivirus. After I verified everything was running smoothly I downloaded
F-prot and removed Mcafee before the weekend and things have also been running
very smooth. Just a note, I had run Declude Antivirus for years without
any issues and I only seem to run into these overflow problems when I introduce
Declude Junkmail.
Since
the major issues I had last week I have made three changes… I have
modified the queue manager to process 100 messages at a time. I have
disabled Declude Junkmail. I have now moved from Mcafee to F-Prot.
The only one of these changes that has ensured that mail does not go into
overflow and backup is the disabling of Declude Junkmail. I’m not
so sure it’s a DNS issue because the Imail spam filters run perfectly
fine, which I am now using in place of Declude. They do not do as good a
job identifying spam, but they are better than nothing.
Things
to think about:
I was
interested in the statement that the mail server can be setup to not bounce
messages if they are detected as SPAM. This still confuses me, because the
BOUNCEONLYIFYOUMUST statement in the $default$.junkmail file appears to
actually bounce the messages. As it is, all my filters are set to WARN
and I don’t understand why changing them to BOUNCEONLYIFYOUMUST will
help.
I do not
believe I have bulk mailers on my system, but I will not discount it.
Does anyone have a good suggestion for determining this? After looking
through my logs and running Imail log analyzer none of my own IP’s stand
out for message counts and the invalid user entries I find are from outside
sources outside of my network. These connections are always denied and
mail does not get processed onto our server. I will think about Declude
HiJack.
I still
want to use Declude Junkmail, so I will test some of the suggestions put into
place such as whitelisting my IP range and disabling outbout virus scanning,
but other than these two suggestions, I don’t see any major changes here
that will affect me.
Will
Friday:
SpamContent 13400
SpamPhrase 19107
SpamFeatures 1283
SpamUrlDomain 35018
LocalDeliver 92331
RemoteDeliver 10526
Saturday:
SpamContent 10089
SpamPhrase 20739
SpamFeatures 1090
SpamUrlDomain 27194
LocalDeliver 83297
RemoteDeliver 9008
Sunday:
SpamContent 10425
SpamPhrase 18741
SpamFeatures 793
SpamUrlDomain 25756
LocalDeliver 81033
RemoteDeliver 10142
-----Original
Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On
Behalf Of Matt
Sent: Friday, July 29, 2005 2:03
PM
To: [email protected]
Subject: Re: [Declude.JunkMail]
Declude Woes
Will,
What you probably have are users bulk-mailing through your server (evidenced by
the GSE files), issues with running both IMail Antispam and Declude, and
choosing a much slower than optimal virus scanner as your one an only engine in
a CPU-challenged environment. There could well be issues with
fragmentation and not enough disks as well.
The reports that you shared are somewhat inconclusive, primarily because
running Declude plus IMail Antispam will skew these numbers. So there is
no telling what the true volume is. If you run Declude, you should turn
IMail Antispam off and just rely on Declude alone.
Your config looks fine except that you should strongly consider spending about
$50 on an F-Prot license and using that instead of McAfee since you are CPU
challenged.
The fact that you are an ISP also speaks volumes to the need of restricting
bulk-mailing among your client base. Not only can they redline your
server, they can also get you blacklisted. Adding a gateway will do
nothing for this part of the issue. You need to look into integrating
Declude Hijack, but be prepared to sort through issues in the week that follows
integrating it.
To save CPU, you could whitelist your userbase in Declude by their IP space by
listing them in your global.cfg. You could also turn off outbound virus scanning
with Declude Virus Pro (not sure if that also applies to Declude Standard).
IMO, this isn't an issue of sizing or a bug in Declude, this is all about
optimizing your configuration where it counts most, and putting in place
reasonable protections so that bulk-mailing isn't allowed to go through your
mail server. Jumping to a gateway will just be another partial fix and it
might not provide enough juice to prevent you from backing up, especially if
the load is primarily the result of customers bulk-mailing through your server.
If you did everything that I identified here, you would save about 1/2 to 2/3
of your current CPU utilization. That might not cut it however since you
were clearly demanding more than 100% before, but I believe it should.
Matt
Will wrote:
Well
thank you to everyone who responded to my need for perspective and assistance.
I
believe I will be in need of a mail gateway if I am to continue to use Declude.
Also,
does anyone know how to configure an Imail/Declude setup to not bounce spam
messages?
Will
-----Original
Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]
On Behalf Of Michael Jaworski
Sent: Friday, July 29, 2005 1:11
PM
To: [email protected]
Subject: RE: [Declude.JunkMail]
Declude Woes
I can second
the need for a gateway defense when under attack. We run a Windows
shop and were being crushed under multiple dictionary attacks for two
domains on a daily basis. I took the daunting task to build our first
Linux box running Postfix. The first box was a tough start though I had a
employee who had Linux experience. We are running Postfix on OpenBSD 3.6 with
MySql for dynamic update ability. (I am still working on grabbing additions,
updates and deletions from SmarterMail admins so we can throw all our domains
in Postfix and update in realtime) After a few weeks we added a second box in
the event the first box went down. The second box was a breeze since it was
basically a duplication. Both mx records now point at the two boxes. The
hardware was old 500Mhz and 1ghz cpu with 512mbs of ram each. The 1ghz is
primary and takes 75% of the load without much effort with plenty of
free memory. The whole setup allows the main server running
SmarterMail/Declude Pro/Sniffer/F-Prot to respond quickly to POP, web mail and
smtp traffic requests.
The
Linus approach only should cost you some time and old equipment as the software
is free. Our experience over the last two years showed it was worth climbing
the short Linux learning cliff. And it is true ... they run forever
One
important note not related to using a gateway: We never bounce spam e-mail
back to the "sender". The backscatter traffic can kill you and
skew your reports.
Michael
Jaworski
Puget Sound Network, Inc.
(206) 217-0400
(800) 599-9485
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]]
On Behalf Of Will
Sent: Friday, July 29, 2005 9:38
AM
To: [email protected]
Subject: RE: [Declude.JunkMail]
Declude Woes
Well,
I’m back at it today.
Yesterday
I disabled Declude early in the day and started working mail back into the
spool directory from the overflow directory. This was a long process, but
by the end of the day I had gone from a backlog of 150,000 files in the spool
and 134,000 in the overflow directory to about 1500 (that includes logs).
During this time I needed to stop and restart the queue manger a number of
time. I did this to allow me to delete all the .gse files, which I
figured would save me time discarding them. However, by the time I got
down to 1500 files and started to watch the spool it started to increase in
size again; climbing to 4000 within a matter of minutes. I stopped and
restarted the queuemanager and these files were then processed. I
verified they were actually getting processed by sending test messages to
myself. At this point I was pretty upset and confused. I looked
through the sys logs and found nothing out of the ordinary, queuemanger would
simply stop. I set all the queuemanager setting back to default and tried
again without luck. I had to stop and restart it every few minutes to get
it to process a few thousand messages. Finally, I purchased an Imail
service agreement and upgraded to 8.21. Magically, it worked. The
queumanger started to deliver messages as soon as they arrived. My
thought immediately went into conspiracy mode. It seems like this has happened
before where we had a perfectly workable solution and something completely
confusing happened and an upgrade magically fixes it!
Anyway…
I re-enabled declude and let it run overnight. Now I have a backlog
again. There are mostly D*.SMD files in the spool right now with all
their delivery Q* files in the overflow directory (*shakes fist at overflow
directory*). Time to start the process again today. I’m
disabling declude to get those messages out and one thing to note, after I have
stopped the smtp server and added smtpd.exe backing into the delivery
application, I still have about 20+ declude.exe processes. I have stopped
and started it again as well as the queuemanager and they are still
there. In fact they are creating more declude.exe processes as I
watch. I’m trying to kill them, but they just keep coming
back… having to restart so I can start processing mail.
We are
an ISP and here are some random examples of some of our Imail daily reports to
give you an idea of what kind of traffic we see:
--
=====================================================
MailPure custom filters for Declude JunkMail Pro.
http://www.mailpure.com/software/
=====================================================
