At 1:02 PM -0700 1999/9/17, Matthew Dillon wrote:

>     Sendmail does not get into trouble with queue files it is able to retire
>     quickly.  Where sendmail gets into trouble is with queue files it ISN'T
>     able to retire quickly.  This is why you *see* 10,000+ files in mqueue
>     at times.  These files build up because a small percentage of mail
>     destinations cannot be delivered to immediately.

        In my experience, sendmail almost always accepts messages a lot 
faster than it can process them, regardless of the mode in which 
sendmail is running.  Foreground, queue-only, background, in my 
experience it doesn't make a whole lot of difference.

        So the input number of files can grow very quickly, with 
deliveries (and mqueue file removals) suffering.

>     The reason sendmail tends to break down with large queue directories has
>     little to do with directory overhead and a lot to do with sendmail's own
>     algorithms.  If you have 50 sendmails running a 10,000 file queue, each
>     of those sendmail processes is essentially scanning the entire queue.

        Yup, that's another very real problem with sendmail, and another 
reason why I really like postfix.

>     If not controlled, this eventually leads to a cascade failure.  The
>     potential for a cascade failure is, in fact, the number one reason for
>     *NOT* running sendmail with background queueing mode turned on.  The
>     best way to avoid a cascade failure is to run the sendmail daemon in
>     queue-only mode with a set fork limit:
>       sendmail -bd -OMaxDaemonChildren=X -ODeliveryMode=q
>     And run the sendmail queue runner separately:
>       sendmail -q1m -OMaxDaemonChildren=Y -OMinQueueAge=1h

        In my experience, you still get messages being accepted a lot 
faster than they're being pushed out.  This is usually made even 
worse when you're delivering to mbox-style mailboxes, where a single 
large message may come in addressed to dozens of recipients, and now 
you might have megabytes of data being read but gigabytes of data 
being written.

        This is another of my pet peeves, where I believe that a 
database-oriented solution that writes just one copy of the message 
and then gives all recipients pointers to it, would help a great deal.

        Of course, there's not anyone like Eric or Wietse to pick on when 
it comes to writing database-oriented local delivery agents.

>      The key issue with any mail server is that bandwidth and transaction
>      useage tends to be low relatively speaking.  A USENET news system
>      almost always has much higher transactional overhead, especially if it
>      is taking several feeds.  A million news messages a day translates to
>      around 10 million protocol transactions for a news box taking 4 feeds.

        However, most of those transactions should be looking up 
message-ids in history or precommit cache databases and then refusing 
the article without it actually being transmitted.

        High transactional rates, yes.  But the actual number of articles 
being received would be on par with a very busy mail server.  If 
you've got a lot of outbound feeds, of course the outbound data rate 
could be a very real problem, but that's a separate issue.

>                                       What you cannot afford to spend time
>      doing in a mail server is scanning the same queue file over and over
>      again, so what you want to optimize for are the 5% of email messages
>      that wind up stuck in the queue for more then a few minutes but usually
>      less then an hour, and then make sure the 1% that stick around past
>      that do not interfere with the processing of those that stick around
>      less.

        You can't really fix sendmail in this regard, although you could 
replace it with a different MTA.

        I guess you could change the implementation methods of the 
underlying filesystem so as to speed up those constant linear sweeps 
of the entire mqueue directory by the queue runners (and by every 
sendmail process that goes to create a file in the mqueue, since they 
have to guarantee that the filename they're creating is unique).

        How you would actually do this is totally beyond me, however.

   These are my opinions -- not to be taken as official Skynet policy
|o| Brad Knowles, <[EMAIL PROTECTED]>            Belgacom Skynet NV/SA |o|
|o| Systems Architect, News & FTP Admin      Rue Col. Bourg, 124   |o|
|o| Phone/Fax: +32-2-706.11.11/12.49         B-1140 Brussels       |o|
|o|                     Belgium               |o|
  Unix is like a wigwam -- no Gates, no Windows, and an Apache inside.
   Unix is very user-friendly.  It's just picky who its friends are.

To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-current" in the body of the message

Reply via email to