|
Sweet. Thanks for the testing and follow up on this,
Matt.
Andrew
8)
I just confirmed that this works properly. If the size limit
is exceeded for the DSN, it strips all attachments out and only sends back the
original headers and the top 20 lines of the original message body. I
set the MaxDSNSize to 100 KB, removed the MS SMTP size limitation, and let
IMail return the 552 error to my MS SMTP gateway.
Beautiful. No
more of these retransmissions need to occur now because of Microsoft's
shortcoming.
Matt
Matt wrote:
Andrew,
I will try the MaxDSNSize hack and see if it does
anything. I do know that on vanilla MS SMTP, the limit is not
defaulted to 10 MB because my IMail is set to 25 MB and this was creating a
lot of issues back when I was configured that way. Basically, every
552 bounce was landing in Badmail due to the sending of the DSN with the
full original message (and the messages in Badmail clearly showed sizes
above 10 MB). Hopefully this is something that can be tweaked without
using Exchange. If this works, it would be a good workaround for the
issues on my MS SMTP gateways where some oversized messages are rebroadcast
over and over again until they expire from their queue.
Regarding the
captured text for the PoolThreadLimit, unfortunately this is somewhat
different from the information on Microsoft's site. It would be good
to determine exactly what is going on, though I do have connection limits
set in MS SMTP that are of a fairly low value, or at least low enough that
setting it to 256 shouldn't be an issue. Note that this key is not
present on a vanilla MS SMTP
setup.
Thanks,
Matt
Colbeck, Andrew wrote:
This may be of some use... These are two of the
things checked by theMicrosoft Exchange Best Practices Tool (sorry, there
is no KB listed):
1) How to restrict the size of the bounce
messages you generate (just like the big boys do with their postfix and
sendmail MTAs, but it's possible that Exchange, not IIS uses this, and
that Exchange just uses this registry key as a good place to put the
value):
MaxDSNSize
not set Send Feedback
The Microsoft® Exchange Server Best Practices
Analyzer Tool reads the following registry entry to determine whether the
maximum size for delivery status notification (DSN) messages has been
set:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSvc\Queuing\MaxDSNSize
The Exchange Server Best Practices Analyzer also
queries the Active Directory® directory service to determine the value of
the serialNumber attribute for all objects with an object class of
msExchExchangeServer. If the string value includes "Version 5.5," the
computer is running Microsoft Exchange Server 5.5. If the string value
includes "Version 6.0," the computer is running Microsoft Exchange 2000
Server. If the string value includes "Version 6.5," the computer is
running Microsoft Exchange Server 2003.
If the Exchange Server Best Practices Analyzer finds
that the MaxDSNSize registry value does not exist on a computer that is
running Exchange 2000 Server or Exchange Server 2003, a warning is
displayed.
The MaxDSNSize registry value is an optional value in
Exchange Server that limits the size of DSNs. This option configures
Exchange to remove attachments if a message cannot be delivered. If you
add this registry value, it is recommended that a value of 10240000 bytes
(10 megabytes) be used.
Note:
The MaxDSNSize registry value was
introduced in Service Pack 2 (SP2) for Exchange 2000 Server, and therefore
requires Exchange 2000 Server SP2 or later versions to work. Additionally,
in Service Pack 1 for Exchange Server 2003 and later versions, a value of
10 megabytes is used in the absence of the MaxDSNSize registry value. If
you want, you can add the MaxDSNSize registry value to override this
default behavior.
If you
enable this option, you can save server and network resources. However,
there are drawbacks to this implementation of Simple Mail Transfer
Protocol (SMTP) attachment stripping. If you enable this option to strip
the attachments from the non-delivery report (NDR), the details that are
necessary to display the notification in the preview pane are also
stripped, and the originator of the message cannot use the Send Again
option. If the originator of the message tries to use the Send Again
option from the NDR, the originator of the message receives the following
error message: Unable to resend the message. The nondelivery report does
not contain sufficient information about the original message. To resend
the message, open it in your Sent Items folder, click the Actions menu,
and click "Resend this message".
However, the
originator of the message cannot resend the message, even by using the
method in the error message.
Important:
This article contains information about
editing the registry. Before you edit the registry, make sure you
understand how to restore the registry if a problem occurs. For
information about how to restore the registry, view the "Restore the
Registry" Help topic in Regedit.exe or
Regedt32.exe.
To
correct this warning
Open a registry editor, such as Regedit.exe or
Regedt32.exe.
Navigate to:
HKLM\System\CurrentControlSet\Services\SMTPSvc
Right-click
SMTPSvc and select New | Key. Name the new key Queuing and leave the class
blank.
Right-click
Queuing and select New | DWORD Value. Name the new value
MaxDSNSize.
In the right
pane, double-click the MaxDSNSize registry value and enter the size limit
(in bytes) that you want in the Value data field. Messages that are larger
than this value that generate an NDR do not return attachments or full
message properties.
Close the
registry editor, and then restart the Simple Mail Transfer Protocol (SMTP)
service for the change to take effect.
For more
information about the MaxDSNSize registry setting, see the following
Microsoft Knowledge Base articles:
2) One possible registry key that would scale up the
number of threads used by IIS (hey, it seems like a good
entry):
The Microsoft® Exchange Server Best Practices
Analyzer Tool reads the following registry entries to determine whether
the maximum number of Internet Information Services (IIS) pool threads has
been modified from the default value:
HKEY_LOCAL_MACHINE\
System\CurrentControlSet\Services\Inetinfo\Parameters\PoolThreadLimit
If the Exchange Server Best
Practices Analyzer finds PoolThreadLimit to be present and
configured with an alternate value, a non-default configuration message is
displayed.
The PoolThreadLimit
registry value specifies the maximum number of input/output (I/O) worker
threads that can be created in the Inetinfo.exe process, which in turn
controls the maximum number of simultaneous connections that can be made
to IIS. Each pool thread watches for a network request and processes it,
either by sending back a static file or by passing the request to an ISAPI
extension DLL (such as ASP.DLL) or to a Common Gateway Interface (CGI)
application. If the ISAPI extension processes a request synchronously and
it takes a long time to process requests, it will tie up the worker
thread, leaving IIS with fewer worker threads to process other requests.
For this reason, well-written ISAPI extensions, such as ASP, implement
their own thread pools, place requests in a queue, and process them
asynchronously with their own threads, so as not to tie up IIS worker
threads.
By default, IIS sets
PoolThreadLimit to:
2 * number of megabytes of RAM
present in the machine
If this value is larger than 256,
it will be clamped to 256. By default, this registry value does not exist.
If this value is present in the registry, it overrides IIS's default
calculation.
Generally, if you find that the
default limit of 256 threads is inadequate, you probably have a poorly
written ISAPI extension tying up IIS worker threads. 256 is a lot of
threads to have active simultaneously and will incur significant overhead
in synchronization and context switching.
The PoolThreadLimit
registry value is a hard limit that includes all IIS worker threads,
including the HTTP, FTP, NNTP, and SMTP services.
Andrew 8)
Sandy, you have such a delicate way with
words...
I am only passing on what I was told by Peter from
VamSoft. There is a whole thread on this where Peter confirmed the
affect on FTP (but not IIS) in VamSoft's o.e.support newsgroup, which I
see you found after sending this message.
I personally would be
surprised to see a fixed (non-tweakable) thread limit, but I can't make
a judgment without some form of verification of your suggestion
considering the guidance that I was given (I'm not the programmer nor
the expert). This server averages only about 2% CPU utilization,
so it would be a shame to have it top out so prematurely. I did
previously search for "CDO_OnArrival thread", but came up with nothing
at all that was useful. Looks like I should have searched for
"SMTPSVC threads" instead since I did come across an article mentioning
the registry parameters that you mentioned. I'll wait for Peter to
reply to your suggestion.
Regarding the MS SMTP tarpitting, you
have to trust me on that, or at least test it yourself before suggesting
that I am wrong here. This was a different issue than that other
server had. The issue with that server was that MS SMTP returns
552 errors in the middle of the DATA, and while many servers support
that, IMail doesn't. I have since convinced people at Ipswitch
that this needs to be changed even if it isn't technically RFC compliant
due to real-world conditions, and the fact that Microsoft's
implementation makes more sense and should be easy enough to
support. This hasn't apparently been worked into the software yet
because they were in the final stages of testing IMail 2006, and this
isn't a quick fix. I expect to see it happen soon though,
hopefully right after they get the major issues worked out with the 2006
release.
The MS SMTP tarpitting issue is one that I came across
under other circumstances. When I had this turned on (not the ORF
version, but the registry hack), I experienced the condition repeatedly
from all sorts of servers where I was the recipient. After
extensive research and testing, I found that it was this registry hack
that was causing the majority of the servers to never recognize the 552
error code, and since all this did was delay the response, one has to
assume that the delay of the 552 error was responsible. With this
turned off (the default), I am still experiencing some issues with
requeued incoming oversized E-mail, but they are much lower in number
and related to the sending of 552 errors in the middle of the DATA
command. Microsoft shouldn't be haphazardly delaying all 5xx
errors, but instead should just be delaying the ones related to invalid
recipients. This was a bad oversight on their part, and I am 100%
positive about the cause. It may be almost unnoticeable to those
that are running this in front of less E-mail addresses and don't
actively monitor their bandwidth like I do. Due to my volume, this
issue became so large that it was eating up many times my normal
incoming bandwidth on a regular basis.
Note that I tried to work
around both MS SMTP 552 issues on incoming E-mail by setting MS SMTP to
accept messages of unlimited size and then configuring IMail to signal
the error, but MS SMTP hears the 552 error that IMail generates and then
tries to bounce the entire oversized message through the configured
Smart Host, which was the same IMail server that wouldn't accept the
message in the first place. Why you would want to send a 50 MB
bounce message is beyond me. I could kludge something together to
bypass the problem, that's not the issue though. My point is that
I have been through this stuff with a fine-toothed
comb.
Matt
Sanford Whiteman wrote:
The exact source of the problem was a limit in threads in MS SMTP,
which is something like 20 per processor (not sure if hyperthreading
counts in this case). . .
This isn't an accurate claim. It sounds like you just haven't tried to
tune MS SMTP appropriately for load/resources/growth. That's fine,
since MS doesn't push you too hard on this, but please do some more
lab work before deciding that MS SMTP is broken.
Look into the following values:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\InetInfo\Parameters\PoolThreadLimit
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\MaxPercentPoolThreads
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\SMTPSVC\Queuing\AdditionalPoolThreadsPerProc
You also need to realize the differences between I/O completion port
architecture, callbacks, and pure multithreading. It's possible to
bottleneck anything, but that's an implementation problem with a given
event sink, not a problem with the overall framework.
. . . this limitation in the MS SMTP architecture makes it
inappropriate for any full scale spam blocking solution such as
Declude unless you have that application ride behind MS SMTP instead
of being plugged into it. . . .
Sorry, man, but that's FUD. I appreciate what you found with FTP in
your config, but let's not jump over the available data.
One other note. I had previously used the Windows registry hack to
enable a native tarpitting feature in MS SMTP with even worse
affects. The built-in tarpitting in MS SMTP will delay all 5xx error
codes by the time that you set, and this included the 552 error used
when messages are over the maximum size. The result of this terrible
oversight is that a fair number of servers will not recognize the
delayed 552 error and will requeue the oversized messages over and
over again, and that eats up a lot of bandwidth real, real quick.
Matt, you _know_ this isn't cause and effect, since you and I both
looked at a specific server _without_ tarpitting enabled that
exhibited the problem with oversize messages. There may be an
additional wrinkle added by ttarpitting, but it's not the root cause
of the problem.
--Sandy
------------------------------------
Sanford Whiteman, Chief Technologist
Broadleaf Systems, a division of
Cypress Integrated Systems, Inc.
e-mail: [EMAIL PROTECTED]
SpamAssassin plugs into Declude!
http://www.imprimia.com/products/software/freeutils/SPAMC32/download/release/
Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases!
http://www.imprimia.com/products/software/freeutils/exchange2aliases/download/release/
http://www.imprimia.com/products/software/freeutils/ldap2aliases/download/release/
---
[This E-mail was scanned for viruses by Declude EVA www.declude.com]
---
This E-mail came from the Declude.JunkMail mailing list. To
unsubscribe, just send an E-mail to [EMAIL PROTECTED], and
type "unsubscribe Declude.JunkMail". The archives can be found
at http://www.mail-archive.com.
|
