Very informative Sandy. Not being a windows admin how do you monitor the async process state, and sockets used?
-----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Sanford Whiteman Sent: Sunday, December 19, 2004 11:57 PM To: Matt Subject: Re[2]: [IMail Forum] Separate SMTP Server - Slightly OT? > Even the thread limit itself is not a practical limit, and combined > with keeping connections open for hours, that can kill a server on a > volume of only about 150,000 messages a day. In my response, I'm going to: 1) nitpick about terminology and rearticulate the current architecture; 2) shamelessly defend Ipswitch's implementation under certain circumstances; and 3) offer a suggestion for working around the architectural limitations, which I too acknowledge. So, if you stick around for the end, there should be no (or little) need to flame. :) 1. The limit is not a thread limit. There's no "thread limit" in SMTPD--certainly not a 60-thread limit. SMTPD is not a fully multithreaded TCP/IP application, where a new worker thread is created for every incoming connection. Rather, there is a very small fixed set of threads (5-6 depending on version), each thread dedicated to a specific global function such as listening for incoming connections, performing AUTHs, etc. SMTPD's connection management, therefore, does not use a multithreaded architecture. Instead, it uses an asynchronous/non-blocking I/O architecture, an alternate method of implementing TCP/IP servers. With multithreading servers, a core process--well, technically, it's actually the primary thread of the process, not the "process" itself--spawns new standalone threads after each TCP/IP socket is connected. With async I/O, all sockets are managed within a single thread, with the OS notifying each open socket when new data is available. It's a strikingly different programming paradigm. Despite the fearful sound of "running within a single thread," async I/O is by no means inherently inferior to multithreading. For I/O-intensive applications with a predictable number of well-behaved client connections and a server processors, async I/O is in fact significantly faster than multithreading. Its superior performance can be laid to the facts that it does not require a single processor to perform context switches between threads, making simultaneous servicing of multiple connections much more efficient, as well as easier on the overall CPU utilization; and that it is generally interrupt-driven, meaning cycles are not wasted waiting for client activity. Yet in that last sentence can be found the Achilles' heel in choosing async I/O for SMTPD in the contemporary SMTP environment. The power of not manually polling sockets for activity, and simply awaiting OS notification, turns into a weakness when you have a significant number of "unclean" client connections (read: spamware) that do not properly close sockets. The app doesn't constantly scan the sockets, so connections that are closeable won't be closed until the OS says so. And here's the real rub: Windows imposes a limit of 64 TCP/IP sockets per thread that it will notify using async I/O. With only one thread actually listening on port 25, there you have your limit of "about 60" concurrent sockets. That, then, summarizes the main _stability_ concern in choosing a single-threaded, async I/O based model for a contemporary SMTP daemon. There are performance concerns as well, one being the fact that multithreading is far more friendly to multiprocessor machines, as new threads will execute across the processor pool, rather than being confined to the processor which launches the listener thread. If you want SMTPD to grab as many of your processors as possible (see below), multithreading would be the way to go. 2. Some rationalizations of the current architecture. Despite the above, I can understand the desire to choose async I/O for the initial architecture, regardless of its current lack of fitness to task. First, async I/O _is_ much faster than multithreading when you have predictable client load and behavior. 64 active sockets was clearly sufficient for the majority of deployments in the "good old days" when (a) MUAs closed connections gracefully, and (b) broadband was scarce enough that the futility of SMTP for file transfer would be apparent enough at the client end that people didn't try to push it (I can't be alone in having noticed that users seem less familiar with FTP than ever, despite a never-greater need for point-to-point file transfer). So, while async I/O may be a bad choice now, it was actually a superior choice originally (as opposed to always having been a poor choice). Second, while async I/O is not ideal for multiprocessor support, it's important to remember that IMail is not solely an SMTP daemon. It's a "horizontally integrated" suite including a mail client (IWEBMSG), standard mail retrieval protocols (POP3 and IMAP4), _and_ an SMTP send/receive/delivery module. So building a heavily multhreaded SMTP daemon would not befit the balanced use of all daemons, as is common in mailbox/hosting environment. And, before the onslaught of dumped or rejected spam, if the SMTP service of your mailbox were being excessively taxed, the IWEBMSG or POP3/IMAP sides would soon follow, so allowing one service to overrrun the others was impractical. Of course, with the increased use of SMTPD only as a dedicated gateway (thanks both to Declude and an overall increase in demand for backup MXs in the past few years), there's now a need to have a daemon that can be set to grab every resource available on the box; yet it's easy to see why this wouldn't have been a consideration in the traditional deployment as a "one-stop" product suite. Third, Ipswitch had in fact been moving, however slowly, toward full multithreading, until whatever management/personnel changes occurred, so it's not like they don't have developers who know the need (I'm sure that Tripp is totally aware). Process-based SMTP32 became multithreaded QUEUEMGR, which was a real leap forward that was probably not appreciated by those who weren't maxing out their send/deliver processes prior to the change. When the QUEUEMGR release was announced, Ipswitch personnel specifically said that SMTPD was next on the list for multithreading. Anyone can readily point out the lack of core development of any substantial sort since QUEUEMGR came out, but the stagnation of SMTPD is likely due to a devaluing of development in _general_, not a _specific_ inattentiveness to this need. It's going to mean a huge overhaul if SMTPD is ever multithreaded, and the fact that anti-spam connection filtering happens after spooling, and all the other crazy functional gaps, doesn't lead me to believe that there's any "planned obsolescence" going on--just, perhaps, accidental suicide. 3. Something to try. If you're having problems with sockets not being closed by the OS for what seems like ages, try this Registry tweak: HKLM_SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\KeepAliveTime REG_DWORD 60000 ( = 1 minute) Some, if not all, abnormally ended client connections should be cleaned up by this setting. The default, before you added this setting, was two hours. --Sandy ------------------------------------ Sanford Whiteman, Chief Technologist Broadleaf Systems, a division of Cypress Integrated Systems, Inc. e-mail: [EMAIL PROTECTED] SpamAssassin plugs into Declude! http://www.mailmage.com/products/software/freeutils/SPAMC32/download/release / Defuse Dictionary Attacks: Turn Exchange or IMail mailboxes into IMail Aliases! http://www.mailmage.com/products/software/freeutils/exchange2aliases/downloa d/release/ http://www.mailmage.com/products/software/freeutils/ldap2aliases/download/re lease/ To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/ To Unsubscribe: http://www.ipswitch.com/support/mailing-lists.html List Archive: http://www.mail-archive.com/imail_forum%40list.ipswitch.com/ Knowledge Base/FAQ: http://www.ipswitch.com/support/IMail/
