Andrew,
Thanks for your notes and their history.
I'm using the following settings right now:
THREADS 30
WAITFORMAIL 500
WAITFORTHREADS 200
WAITBETWEENTHREADS 100
WINSOCKCLEANUP OFF
INVITEFIX ON
AUTOREVIEW ON
There are a few reasons for trying these values.
THREADS 30 - I'm pretty confident that dual 3.2 Ghz
Xeons and RAID can only handle 30 threads with average messages. In
reality, one single message can spike the system to 100%, but these are
uncommon. I figure that if I open this up too wide and I am dealing
with a backup or something, launching more threads when at 100% CPU
utilization will actually slow the system down. This was the same with
2.x and before. There is added overhead to managing threads and you
don't want that to happen on top of 100% CPU utilization. I am going
to back up my server later tonight to see if I can't find what the
magic number is since I don't want to be below that magic number, and
it would probably be best to be a little above it.
WAITFORMAIL 500 - On my server, this never kicks in, but if it
did, it wouldn't make sense to delay for too long because I could build
up messages. A half second seems good.
WAITFORTHREADS 200 - This apparently kicks in only when I
reach my thread limit; sort of like a throttle. I don't want it to be
too long because this should only happen when I am hammered, but it is
wise not to keep hammering when you are at 100%. Sort of a mixed bag
choice here.
WAITBETWEENTHREADS 100 - I see this setting as being the
biggest issue with sizing a server. Setting it at 100 ms means that I
can only handle 10 messages per second, and this establishes an upper
limit for what the server can do. I currently average about 5
messages per second coming from my gateways at peak hours, so I figured
that to be safe, I should double that value.
INVITEFIX ON - I have it on because it comes on by default and
I don't know any better. I know nothing about the cause for needing
this outside of brief comments. It seems strange that my Declude setup
could ruin an invitation unless I was using footers. If this is only
triggered by footer use, I would like to know so that I could turn it
off. I would imagine that this causes extra load to do the check.
AUTOREVIEW ON - I have this on for the same reason that Andrew
pointed out. When I restart Decludeproc, messages land in my review
folder, and I don't wish to keep manually fishing things out. If there
is an issue with looping, it would be wise for Declude to make this
only trigger say every 15 minutes instead of more regularly.
Feel free to add to this if you want.
Matt
Colbeck, Andrew wrote:
I'd second that... on both the
observed behaviour and the request for documentation.
I'm attaching my
highly commented declude.cfg as a reasonable sample.
Andrew 8)
David,
That did the trick. I can't even see any messages in my proc folder
any more. I might suggest adding your explanation to the comments in
the file just in case others feel the need to turn this on like I did.
I recalled the issues from the list and I turned it on because I didn't
want the possibility of DNS crapping out and the leakage that this
would cause.
Here's a screen cap of what my processor graph looks like now:
Thanks,
Matt
David Barker wrote:
The purpose of WINSOCKCLEANUP ON is to reset the winsock, what
happens when using this setting is that when the \proc directory hit 0
decludeproc will finish processing all the messages in the \work before
checking the \proc again. As WINSOCKCLEANUP is to be used only by those who
experience DNS issues I would suggest running your tests again with
WINSOCKCLEANUP commented out and see how the behavior differs. Also having
the WAITFORMAIL to low can cause the CPU to process very high as it is
constantly checking the \proc I would suggest a minimum of 500-1000
David B
www.declude.com
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Matt
Sent: Monday, May 22, 2006 8:12 PM
To: Declude.JunkMail@declude.com
Subject: Re: [Declude.JunkMail] Experience with 4.x
Darrell,
I put up two Windows Explorer windows side-by-side under normal volume
and the pattern was consistent where the proc folder grows while the
work folder shrinks until the work folder hits zero at which point the
proc folder empties out and everything lands in work and then the
pattern repeats with proc growing while work shrinks.
My settings are as follows:
THREADS 50
WAITFORMAIL 100
WAITFORTHREADS 10
WAITBETWEENTHREADS 50
WINSOCKCLEANUP ON
AUTOREVIEW ON
INVITEFIX ON
Matt
Darrell ([EMAIL PROTECTED]) wrote:
It's a faulty design that leaves more than half a server's CPU
capacity unused due to the mere fact that they wait for all threads
to complete before moving in a new batch.
I can't speak to what you see on your server, but that is not how it
is running on my server. I just double checked again to make sure I
am not crazy, but as I watch the thread count on my server
(decludeproc) the threads fluctuate between 7 - 30 ( threads currently
set to 50). It is not uncommon to see the threads move as follow:
11,8,10,7,15,.... While I was watching it I never seen a case where
it went down low enough for the WAITFORMAIL setting to kick in.
Watching the proc/work directory you can see files moving in and out,
but never really emptying out. Its possible what I am seeing is an
anomaly or maybe I am interpreting it wrong.
Maybe David can comment on this.
Darrell
------------------------------------------------------------------------
invURIBL - Intelligent URI filtering plug-in for Declude, mxGuard, and
ORF. Stop spam at the source the spamvertised domain. More effective
than traditional RBL's. Try it today - http://www.invariantsystems.com
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
|