qmail Digest 14 Apr 1999 10:00:01 -0000 Issue 610
Topics (messages 24216 through 24274):
qmail-ldap run error
24216 by: Van Liedekerke Franky <[EMAIL PROTECTED]>
messages ... have been bouncing"
24217 by: Ralf Nagel <[EMAIL PROTECTED]>
24225 by: Harald Hanche-Olsen <[EMAIL PROTECTED]>
Show: ...... 10K of 100K
24218 by: Attila Csosz <[EMAIL PROTECTED]>
24235 by: "Fred Lindberg" <[EMAIL PROTECTED]>
24244 by: Peter van Dijk <[EMAIL PROTECTED]>
Non-ASCII-characters in Header
24219 by: Dave Sill <[EMAIL PROTECTED]>
24220 by: Juergen Schubert <[EMAIL PROTECTED]>
24221 by: [EMAIL PROTECTED]
24222 by: Stefan Paletta <[EMAIL PROTECTED]>
24223 by: Juergen Schubert <[EMAIL PROTECTED]>
24227 by: "Timothy L. Mayo" <[EMAIL PROTECTED]>
24228 by: "Fred Lindberg" <[EMAIL PROTECTED]>
24232 by: Greg Owen {gowen} <[EMAIL PROTECTED]>
[Q] qmail speed "again"
24224 by: Dave Sill <[EMAIL PROTECTED]>
24234 by: Marc Slemko <[EMAIL PROTECTED]>
24236 by: Mark Delany <[EMAIL PROTECTED]>
24239 by: Dave Sill <[EMAIL PROTECTED]>
24251 by: Keith Burdis <[EMAIL PROTECTED]>
24254 by: Stefan Paletta <[EMAIL PROTECTED]>
24255 by: [EMAIL PROTECTED]
24256 by: Richard Letts <[EMAIL PROTECTED]>
24257 by: "Craig I. Hagan" <[EMAIL PROTECTED]>
24265 by: Silver CHEN <[EMAIL PROTECTED]>
[Q] qmail speed for 'bulk' mails - thanks for the response!
24226 by: Silver CHEN <[EMAIL PROTECTED]>
24231 by: Dave Sill <[EMAIL PROTECTED]>
24237 by: Peter van Dijk <[EMAIL PROTECTED]>
24241 by: Dave Sill <[EMAIL PROTECTED]>
24245 by: Peter van Dijk <[EMAIL PROTECTED]>
24250 by: Dave Sill <[EMAIL PROTECTED]>
24252 by: Peter van Dijk <[EMAIL PROTECTED]>
helping qmail vs. lame MTAs
24229 by: [EMAIL PROTECTED] (John R. Levine)
talk on qmail
24230 by: Marlon Anthony Abao <[EMAIL PROTECTED]>
24233 by: Greg Owen {gowen} <[EMAIL PROTECTED]>
24240 by: Dave Sill <[EMAIL PROTECTED]>
24242 by: "Greg Owen {gowen}" <[EMAIL PROTECTED]>
IRIX NFS, qmail
24238 by: Doug McClure <[EMAIL PROTECTED]>
24269 by: [EMAIL PROTECTED] (David Lindes)
24270 by: Mark Delany <[EMAIL PROTECTED]>
24271 by: [EMAIL PROTECTED] (David Lindes)
24272 by: Marlon Anthony Abao <[EMAIL PROTECTED]>
Virtual Users (?)
24243 by: "Gary Stewart" <[EMAIL PROTECTED]>
FW: Web Interface to Qmail on Linux
24246 by: "Matthew Kaing" <[EMAIL PROTECTED]>
24247 by: Peter van Dijk <[EMAIL PROTECTED]>
24248 by: Peter Gradwell <[EMAIL PROTECTED]>
24264 by: "Sam" <[EMAIL PROTECTED]>
web inteface to Qmail
24249 by: Mark Swanson <[EMAIL PROTECTED]>
qmail/ezmlm writing strange envelope From?
24253 by: [EMAIL PROTECTED] (David Lindes)
Virtual domains and rcpthosts... PROBLEM!
24258 by: Juan Carlos Castro y Castro <[EMAIL PROTECTED]>
24259 by: Mark Delany <[EMAIL PROTECTED]>
24260 by: Juan Carlos Castro y Castro <[EMAIL PROTECTED]>
24262 by: Juan Carlos Castro y Castro <[EMAIL PROTECTED]>
24263 by: Mark Delany <[EMAIL PROTECTED]>
cyclog vs. syslog (was: Queue limit question)
24261 by: Arnaldo Mandel <[EMAIL PROTECTED]>
Qmail queue stuck all the time, any ideas ?
24266 by: Dinesh Punjabi <[EMAIL PROTECTED]>
24268 by: Mark Delany <[EMAIL PROTECTED]>
qmail speed
24267 by: [EMAIL PROTECTED] (David Lindes)
POP security question
24273 by: Joe Junkin <[EMAIL PROTECTED]>
24274 by: Peter van Dijk <[EMAIL PROTECTED]>
Administrivia:
To subscribe to the digest, e-mail:
[EMAIL PROTECTED]
To unsubscribe from the digest, e-mail:
[EMAIL PROTECTED]
To bug my human owner, e-mail:
[EMAIL PROTECTED]
To post to the list, e-mail:
[EMAIL PROTECTED]
----------------------------------------------------------------------
sounds like a library problem to me...
> ----------
> From: BoLiang[SMTP:[EMAIL PROTECTED]]
> Sent: Thursday, February 18, 1999 1:35 PM
> To: [EMAIL PROTECTED]
> Subject: qmail-ldap run error
>
> Hi
>
> After I setup the qmail-ldap package on a Redhat5.2,
> I run it by:
>
> # csh -cf '/var/qmail/rc &'
>
> and I got a error message in the maillog file like:
>
> qmail: 924003326.292702 alert: cannot start: hath the
> daemon spawn no fire?
>
>
> Does anyone has any good idea to fix it?
>
>
> Thanks a lot
>
> ----
> BoLiang [EMAIL PROTECTED]
>
Hi,
I just received above warning from the qmail list server.
Some of the messages (from April 1st) have been bouncing:
123.123.123.123 does not like recipient
remote host said [...] we do not relay
I forwarded the warning message to my provider asking for
an explanation.
This is the answer.
My provider has set up a primary mail server and several backup
servers. On of the backup servers is located in the domain of my
provider's provider! For any reason the qmail list server attempted a
delivery to the mail server with the lowest priority first
(that foreign server) and failed.
And according to my provider the primary within their own domain
was up and running that date. But in his next sentence he said,
that he has (just) adjusted the DNS entries.
This sounds a little bit strange to me, but I don't have the
necessary background.
Is such a scenario possible or not? Or could it be a malfunction
of the list server?
I don't like to be removed from mail lists due to badly configured
servers at my provider...
Thanks...Ralf
+ Ralf Nagel <[EMAIL PROTECTED]>:
| I just received above warning from the qmail list server.
| Some of the messages (from April 1st) have been bouncing:
|
| 123.123.123.123 does not like recipient
| remote host said [...] we do not relay
|
| I forwarded the warning message to my provider asking for
| an explanation.
|
| This is the answer.
|
| My provider has set up a primary mail server and several backup
| servers. On of the backup servers is located in the domain of my
| provider's provider! For any reason the qmail list server attempted
| a delivery to the mail server with the lowest priority first (that
| foreign server) and failed.
Well, under no circumstance is it acceptable for a host which is
listed as an MX for a domain (no matter what its priority) to refuse
relaying for that domain.
| And according to my provider the primary within their own domain
| was up and running that date.
There are other reasons why another MX may be used: Perhaps some part
of the network was down, and the primary MX could not be reached.
- Harald
Is there any console based tool which can help me to see the advance
of sending a large file? Like the wget program.
I send sometimes large files and I'd like to see the advance of the process of
the sending my mail.
Thanks
Attila
RedHat 5.2/Kernel 2.0.36/ppp only
On Tue, 13 Apr 1999 14:46:18 +0200, Attila Csosz wrote:
>Is there any console based tool which can help me to see the advance
>of sending a large file? Like the wget program.
>I send sometimes large files and I'd like to see the advance of the process of
>the sending my mail.
No, but if you have a link that is that slow, consider using some
compressed protocol for transferring your mail to a smarthost. Look
into DJB's serialmail package.
qmail is not a GUI program, but a daemon to work in the background. To
implement what you want, you'd have to hack qmail-remote which would
have to output data to a file/socket/whatever that you could monitor.
Since qmail uses several qmail-remote in parallel, the output would
have to be separated, etc.
Alternatively, use a GUI MUA that sends to your remote host directly
via SMTP. There are several that have some sort of progress indicator.
You can probably also [linux] monitor net traffic and set up something
that counts traffic to the recipient SMTP port from "go" and displays
progress.
-Sincerely, Fred
(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)
On Tue, Apr 13, 1999 at 11:08:18AM -0500, Fred Lindberg wrote:
> On Tue, 13 Apr 1999 14:46:18 +0200, Attila Csosz wrote:
>
> >Is there any console based tool which can help me to see the advance
> >of sending a large file? Like the wget program.
> >I send sometimes large files and I'd like to see the advance of the process of
> >the sending my mail.
>
> You can probably also [linux] monitor net traffic and set up something
> that counts traffic to the recipient SMTP port from "go" and displays
> progress.
I do occasionally use iptraf for just that :)
Greetz, Peter
--
| 'He broke my heart, | Peter van Dijk |
I broke his neck' | [EMAIL PROTECTED] |
nognixz - As the sun | Hardbeat@ircnet - #cistron/#linux.nl |
| Hardbeat@undernet - #groningen/#kinkfm/#vdh |
Juergen Schubert <[EMAIL PROTECTED]> wrote:
>
>I'm searching for a patch which lets qmail to accept mails with characters
>>126 ( e.g. german umlauts ) in the mail headers. I searched all archives
>but it seems no one didn't ask for this up to now.
I just sent myself a test message with such characters in the subject
and qmail didn't complain. Please provide an example bounce message
demonstrating the problem.
-Dave
Hello Dave,
> I just sent myself a test message with such characters in the subject
>and qmail didn't complain. Please provide an example bounce message
>demonstrating the problem.
This is a sample error message. I think your XEmacs is a good one and
encodes the Subject MIME compliant.
-------------------------------
Hi. This is the qmail-send program at janus.pks-software.de.
I'm afraid I wasn't able to deliver your message to the following addresses.
This is a permanent error; I've given up. Sorry it didn't work out.
<[EMAIL PROTECTED]>:
kaufmann: Message contains non-ASCII characters in headers
--- Below this line is a copy of the message.
Return-Path: <[EMAIL PROTECTED]>
Received: (qmail 18379 invoked by alias); 13 Apr 1999 09:51:29 -0000
Delivered-To: [EMAIL PROTECTED]
Received: (qmail 18376 invoked from network); 13 Apr 1999 09:51:29 -0000
Received: from sj.pks-software.de (HELO sj) (194.233.115.194)
by janus.pks-software.de with SMTP; 13 Apr 1999 09:51:29 -0000
Message-Id: <4.2.0.32.19990413115124.01b40f00@janus>
X-Sender: schubert@janus
X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.32 (Beta)
Date: Tue, 13 Apr 1999 11:51:29 +0200
To: [EMAIL PROTECTED]
From: Juergen Schubert <[EMAIL PROTECTED]>
Subject: f�r
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
-----------------------------
In this case the Subject contains an illegal character.
Regards
Juergen
On Tue, 13 Apr 1999, Juergen Schubert wrote:
[snip]
> This is a sample error message. I think your XEmacs is a good one and
> encodes the Subject MIME compliant.
[snip]
> Date: Tue, 13 Apr 1999 11:51:29 +0200
> To: [EMAIL PROTECTED]
> From: Juergen Schubert <[EMAIL PROTECTED]>
> Subject: f�r
The subject looks to me like 'f', a-with-umlaut, 'r'. Dumped from the raw
mailbox, it looks like a '=E4' for the a-with-umlaut, but it's not marked
with the RFC-whatever-it-is prefix saying that it's encoded. I haven't
read the relevant RFC recently enough to remember if this is permitted.
> Mime-Version: 1.0
> Content-Type: text/plain; charset="us-ascii"
>
> -----------------------------
>
> In this case the Subject contains an illegal character.
>
> Regards
> Juergen
>
>
>
--
"Life is much too important to be taken seriously."
Thomas Erskine <[EMAIL PROTECTED]> (613) 998-2836
Juergen Schubert wrote/schrieb/scribsit:
> Hi. This is the qmail-send program at janus.pks-software.de.
> I'm afraid I wasn't able to deliver your message to the following addresses.
> This is a permanent error; I've given up. Sorry it didn't work out.
>
> <[EMAIL PROTECTED]>:
> kaufmann: Message contains non-ASCII characters in headers
This is not an error message from qmail.
Is there anything in the .qmail file that controls this address?
Stefan
Hello Stefan,
you gave me the final tip, thank you very much!
> This is not an error message from qmail.
>Is there anything in the .qmail file that controls this address?
It's the deliver from the cyrus-IMAPD which causes the troubles and I
blamed qmail instead, my fault :-(
But I think this will help me solving the problem much faster now :-)
Thanks to all of you,
Juergen
qmail does NOT reject messages with invalid characters in the headers. IT
DOES NOT PARSE THE HEADERS.
Are you using cyrus for IMAP service. Cyrus does this by default.
Again, this error is NOT coming from qmail, it is coming from whatever you
are using to do the final delivery.
On Tue, 13 Apr 1999, Juergen Schubert wrote:
> Hello Dave,
>
> > I just sent myself a test message with such characters in the subject
> >and qmail didn't complain. Please provide an example bounce message
> >demonstrating the problem.
>
> This is a sample error message. I think your XEmacs is a good one and
> encodes the Subject MIME compliant.
>
> -------------------------------
>
> Hi. This is the qmail-send program at janus.pks-software.de.
> I'm afraid I wasn't able to deliver your message to the following addresses.
> This is a permanent error; I've given up. Sorry it didn't work out.
>
> <[EMAIL PROTECTED]>:
> kaufmann: Message contains non-ASCII characters in headers
>
> --- Below this line is a copy of the message.
>
> Return-Path: <[EMAIL PROTECTED]>
> Received: (qmail 18379 invoked by alias); 13 Apr 1999 09:51:29 -0000
> Delivered-To: [EMAIL PROTECTED]
> Received: (qmail 18376 invoked from network); 13 Apr 1999 09:51:29 -0000
> Received: from sj.pks-software.de (HELO sj) (194.233.115.194)
> by janus.pks-software.de with SMTP; 13 Apr 1999 09:51:29 -0000
> Message-Id: <4.2.0.32.19990413115124.01b40f00@janus>
> X-Sender: schubert@janus
> X-Mailer: QUALCOMM Windows Eudora Pro Version 4.2.0.32 (Beta)
> Date: Tue, 13 Apr 1999 11:51:29 +0200
> To: [EMAIL PROTECTED]
> From: Juergen Schubert <[EMAIL PROTECTED]>
> Subject: f�r
> Mime-Version: 1.0
> Content-Type: text/plain; charset="us-ascii"
>
> -----------------------------
>
> In this case the Subject contains an illegal character.
>
> Regards
> Juergen
>
>
>
---------------------------------
Timothy L. Mayo mailto:[EMAIL PROTECTED]
Senior Systems Administrator
localconnect(sm)
http://www.localconnect.net/
The National Business Network Inc. http://www.nb.net/
One Monroeville Center, Suite 850
Monroeville, PA 15146
(412) 810-8888 Phone
(412) 810-8886 Fax
On Tue, 13 Apr 1999 09:37:18 +0200, Juergen Schubert wrote:
>I'm searching for a patch which lets qmail to accept mails with characters
>>126 ( e.g. german umlauts ) in the mail headers. I searched all archives
>but it seems no one didn't ask for this up to now.
Please post an actual bounce message to the list.
qmail does not reject such messages. Other mailers (exim - depends on
config) do at the SMTP level, which is why your report comes from
qmail.
Here's the scoop:
Some people have decided that all that is not explicitly allowed is
forbidden. To twart SPAM, the check the incoming message at the SMTP
level with certain rules (no From: header => SPAM). As a part of this,
they do a syntax check. Usually, they are in the US and blissfully
unaware of characters that fall into the >127 category.
DJB/qmail has a more intelligent philosophy. Be strict on sending,
liberal on receiving, EXCEPT when it may corrupt the message (e.g. bare
LF handling). Also, they realize the common use of >127 characters in
headers (comments/names - it is trictly forbidden in addresses).
This causes the MTA to reject mail with:
1. Umlauts in headers.
2. Badly formatted headers, even e.g. "From: <me@host>".
3. Messages with e.g. CC: address, but not "To: address".
etc, depending on configuration.
What to do:
mail the postmaster at the site and Cc: the customer explaining what it
happening and why. Hopefully, the MTA programmers will build around
this common (although not explictly rfc822 allowed) use, and still be
able to apply whatever spam rules are desired.
PS: The real joke is all the MX out there that reply "we do not relay".
Their customers get mail intermittently and nobody understands why. On
mailing lists I run, this is maybe half of the bounces. Probably due to
installing a new sendmail version and not knowing what they do ;-)
-Sincerely, Fred
(Frederik Lindberg, Infectious Diseases, WashU, St. Louis, MO, USA)
On Tue, 13 Apr 1999, Juergen Schubert wrote:
> you gave me the final tip, thank you very much!
>
> > This is not an error message from qmail.
> >Is there anything in the .qmail file that controls this address?
>
> It's the deliver from the cyrus-IMAPD which causes the troubles and I
> blamed qmail instead, my fault :-(
>
> But I think this will help me solving the problem much faster now :-)
This has come up repeatedly on the cyrus mail list. If you can
attach to their IMAP archive of the list, you can probably find it without
too much trouble. There's a simple fix, just a few lines of code, that
I've seen posted repeatedly.
--
gowen -- Greg Owen -- [EMAIL PROTECTED]
I'm replying to several messages here (see References), but I'm not
going to bother attributing each quote.
>> >qmail will always be faster than sendmail [unless you send one message
>> >to a large number of addresses on the same remote host].
>>
>> No, qmail will usually win here, too, because sendmail serializes.
>> Sendmail only wins when the message is huge.
>
>Sendmail will win if you use multiple rcpt-to's.
No, SMTP requires too many round trips per recipient.
>> >qmail will always be faster than sendmail [unless you send one message
>> >to a large number of addresses on the same remote host].
>>
>> No, qmail will usually win here, too, because sendmail serializes.
>> Sendmail only wins when the message is huge.
>
>Actually, if you are unfortunate enough to have a list of addresses sorted
>by the right side of the @, qmail can be a big loser here. This is
>because it will completely overload many remote hosts if there are a bunch
>of recipients. eg. concurrencyremote = 120, you have 200 users
>@somedomain, qmail will sit there with 120 connections to somedomain's
>mailserver open while they all crawl along because somedomain can't handle
>120 connections at once.
somedomain is poorly configured. Should qmail assume all sites are
poorly configured? Should properly configured sites suffer because
some sites are poorly run?
>qmail is great that way at inflicting remote DoS attacks against other
>mailers.
That's loaded language. If you want constructive debate, you might
want to avoid that.
>I don't know? Why does "qmail" accept connections that it cannot handle?
Mine doesn't. I restrict it via tcpserver to an acceptable,
conservative number of connections that, in my experience, the system
will be able to handle in most foreseeable circumstances.
>This is great in theory. In practice, does your tcpserver setup
>automatically start refusing connections when you're low on queue space or
>when you know the load will be too high? I bet it doesn't.
No, but my tcpserver connection limit is sufficiently conservative
that it can almost always handle a full compliment of SMTP clients. If
it can't, so what? Chances are, a few too many SMTP connections when
the system is hosed are the least of my worries.
>The reason why it doesn't isn't because it's poorly designed. It's
>because it's hard to do this. You have to be partially psychic in order
>to always catch it.
It's hard (impossible?) to dynamically determine if the system has
sufficient resources to handle a request before accepting the
connection, but, in practice, I've never found it necessary to do
that. Even on a mail server, setting the connection limit
conservatively has always worked fine.
>> Only if they are silly enough to accept more connections than they can
>> handle. :) One of the things a sys admin is "supposed" to do is tune his
>> machines for performance. If you cannot limit the number of connections
>> you will accept to something your system can handle, you need to re-think
>> your setup.
>
>Erm... you just described a classic DoS attack. You put a limit of x
>connections in. One remote system uses all or nearly all of them. No one
>else can connect.
I would call this inadvertant, temporary denial of service, at
worst. It's not an attack, IMO, unless it's intentional and
sustained. Since qmail only sends one message per connection, other
senders are going to have plenty of opportunities to grab
connections. And many of them (e.g., sendmail) will hold onto their
connection until they've dumped their entire load. Enough of them, and
they'll be "denying service" to any other senders.
>> You may have a point here. Is there a well-defined rubric within which
>> we can assert, "It is ill-mannered to consume all available
>> connections to a remote server, just because those services are
>> needed?" Could be, I suppose--that's a question for admins.
>
>The point is that, in a lot of cases, they aren't needed.
>
>If you have 500000 messages to go out with 100 messages to each of 5000
>hosts, the claim that it is necessary to open 100 connections at once to
>any single remote server is obviously wrong.
The claim that qmail forces you to do that is also obviously wrong. IF
you don't want that to happen, don't sort your list by MX.
>Heck, even if you had 500000 messages to go to 1 host that doesn't mean
>you have to open 500000 connections (well, bounded only by your local
>configuration and qmail's limits) to the remote host.
Of course not. What's your point? 256 is << 500000. If I've got 500000
different messages to deliver to a single site, I sure as hell want to
use as many simultaneous connections as I can. I assume the receiving
site will behave responsibly and only accept as many as it can handle.
If the don't, that's their problem.
-Dave
On Tue, 13 Apr 1999, Dave Sill wrote:
> >> >qmail will always be faster than sendmail [unless you send one message
> >> >to a large number of addresses on the same remote host].
> >>
> >> No, qmail will usually win here, too, because sendmail serializes.
> >> Sendmail only wins when the message is huge.
> >
> >Actually, if you are unfortunate enough to have a list of addresses sorted
> >by the right side of the @, qmail can be a big loser here. This is
> >because it will completely overload many remote hosts if there are a bunch
> >of recipients. eg. concurrencyremote = 120, you have 200 users
> >@somedomain, qmail will sit there with 120 connections to somedomain's
> >mailserver open while they all crawl along because somedomain can't handle
> >120 connections at once.
>
> somedomain is poorly configured. Should qmail assume all sites are
> poorly configured? Should properly configured sites suffer because
> some sites are poorly run?
Geesh, this sort of crazy idealism is why qmail gets a bad name.
Please tell me how I can configure qmail so that it isn't "poorly
configured". Please give me an example of how to set it up so that a
remote site can open as many connections as it wants (which you think it
should be able to do) without monopolizing the system.
On top of that, you are still completely ignore the fact that hammering
each remote host as hard as possible, in turn, results in taking a lot
longer to deliver all the mail than if you nicely avoid hammering each
host. Part of this is due to qmail's silly 256 concurrencyremote limit,
which makes connections from the sender a very valuable commodity when
trying to send a lot of mail.
[...]
> >I don't know? Why does "qmail" accept connections that it cannot handle?
>
> Mine doesn't. I restrict it via tcpserver to an acceptable,
> conservative number of connections that, in my experience, the system
> will be able to handle in most foreseeable circumstances.
Then do you really think it is appropriate for one remote system to
monopolize your mailer for no reason? I consider that abusive. You can
go on about "oh, there is really no way to know what you can do so you
just have to try to blast as much as you can" but that ignores the basic
principles that have worked behind all internet services for a heck of a
long time: just because you _can_ do something, doesn't mean you should do
it if it isn't a friendly thing to do.
> >This is great in theory. In practice, does your tcpserver setup
> >automatically start refusing connections when you're low on queue space or
> >when you know the load will be too high? I bet it doesn't.
>
> No, but my tcpserver connection limit is sufficiently conservative
> that it can almost always handle a full compliment of SMTP clients. If
> it can't, so what? Chances are, a few too many SMTP connections when
> the system is hosed are the least of my worries.
"if I can't accept any mail from any other systems because one system
decided to flood me with a huge amount just beacuse it thought it would be
cool to schedule deliveries that way, then it isn't a big deal"
Erm... I don't think that way.
[...]
> >> You may have a point here. Is there a well-defined rubric within which
> >> we can assert, "It is ill-mannered to consume all available
> >> connections to a remote server, just because those services are
> >> needed?" Could be, I suppose--that's a question for admins.
> >
> >The point is that, in a lot of cases, they aren't needed.
> >
> >If you have 500000 messages to go out with 100 messages to each of 5000
> >hosts, the claim that it is necessary to open 100 connections at once to
> >any single remote server is obviously wrong.
>
> The claim that qmail forces you to do that is also obviously wrong. IF
> you don't want that to happen, don't sort your list by MX.
My claim is that qmail's behaviour ranges from extremely unfriendly to
abusive. It is easy to end up with addresses sorted by hostname for a
variety of reasons. It is a flaw in qmail that, if the address list is
sorted in such a way, it takes ~twice as long, in my experience, to send
out a large quantity of mail.
> >Heck, even if you had 500000 messages to go to 1 host that doesn't mean
> >you have to open 500000 connections (well, bounded only by your local
> >configuration and qmail's limits) to the remote host.
>
> Of course not. What's your point? 256 is << 500000. If I've got 500000
> different messages to deliver to a single site, I sure as hell want to
> use as many simultaneous connections as I can. I assume the receiving
You just said that, if qmail didn't have an arbitrary 256 simultaneous
connection limit and if your machine could handle it, then you consider it
perfectly acceptable to try to open 500000 simultaneous connections.
> site will behave responsibly and only accept as many as it can handle.
> If the don't, that's their problem.
That sums up the problems with your attitude quite nicely, and the
problems with far too many "qmail idealists" that are more concernend
about proclaiming qmail to be the greatest thing on earth than about
actually running functional and interoperable mail systems.
It is obvious that you aren't interested in real life problems or actually
using qmail in real life, but only on insisting that qmail is flawless, so
there isn't too much point in continuing this discussion.
At 08:42 AM Tuesday 4/13/99, Marc Slemko wrote:
>On Tue, 13 Apr 1999, Dave Sill wrote:
>
>> >> >qmail will always be faster than sendmail [unless you send one message
>> >> >to a large number of addresses on the same remote host].
>> >>
>> >> No, qmail will usually win here, too, because sendmail serializes.
>> >> Sendmail only wins when the message is huge.
>> >
>> >Actually, if you are unfortunate enough to have a list of addresses sorted
>> >by the right side of the @, qmail can be a big loser here. This is
>> >because it will completely overload many remote hosts if there are a bunch
>> >of recipients. eg. concurrencyremote = 120, you have 200 users
>> >@somedomain, qmail will sit there with 120 connections to somedomain's
>> >mailserver open while they all crawl along because somedomain can't handle
>> >120 connections at once.
>>
>> somedomain is poorly configured. Should qmail assume all sites are
>> poorly configured? Should properly configured sites suffer because
>> some sites are poorly run?
>
>Geesh, this sort of crazy idealism is why qmail gets a bad name.
It may well be. Your trolling is pretty much the other side of the coin.
It'd be nice if you added value rather than repeated what has been said on
this list a number of times before.
Much of what you say is based on speculation rather than fact. And what
little useful discussion you have to contribute is loaded with words like
"crazy", "idealism", "silly", "hammering", "abusive", "extremely unfriendly",
"flaw", etc. Do you seriously expect people to listen to you when you
degenerate into that sort of language?
Furthermore, to claim that we are not interested in running "functional and
interoperable mail systems" just tells us that you have no clue about the
people on this list and what they do. What did you actually want to achieve
with this sort of inflamatory remark? Surely not constructive discussion?
As a small note, the systems I have setup running qmail have probably
delivered well over 500+ million mails to just about every domain on the
planet. Others on this list have done much more, so saying we have no
interest in functional mail systems is a load or rubbish at best and clear
bait at worst.
In short, we've heard your story already, ok? In the spirit of publicly
available sources, either say something new, change qmail in ways you think
are better and share it with the rest of the world or be prepared to be
recognized as yet another "whinger" who finds it easier to complain than do
something.
So what's it to be? Are you more interested in the easy game of flame or
are you actually interested in doing something useful?
Regards.
Marc Slemko <[EMAIL PROTECTED]> wrote:
>
>...Please give me an example of how to set it up so that a
>remote site can open as many connections as it wants (which you think it
>should be able to do) without monopolizing the system.
I don't care if a remote site uses all available SMTP connections if:
o it doesn't "hog" connections, and
o it's not sending spam.
qmail doesn't hog connections because it only sends one message per
connection.
Spam is a separate problem with, IMHO, no technical solution.
>On top of that, you are still completely ignore the fact that hammering
>each remote host as hard as possible, in turn, results in taking a lot
>longer to deliver all the mail than if you nicely avoid hammering each
>host.
Only if the remote host accepts too many connections. Senders can
"hammer" my qmails all they want: I've pre-throttled them to a level
they can sustain. It's only common sense. Arguments like "we never had
to configure our MTA's intelligently before, so if they break now it's
your fault" just don't work.
>Part of this is due to qmail's silly 256 concurrencyremote limit,
>which makes connections from the sender a very valuable commodity when
>trying to send a lot of mail.
I find it very hard to hit that "silly" limit, even on my busiest,
fastest server. I don't see how raising the limit would help
dramatically. And I'd expect Chicken Little's to raise a big stink
about qmails capable of opening thousands of simultanenous
connections. Nonetheless, hardware's getting faster and I expect qmail
2.0 will raise this limit.
>Then do you really think it is appropriate for one remote system to
>monopolize your mailer for no reason?
No reason? No. For the reason of sending me mail? Yes.
>I consider that abusive. You can
>go on about "oh, there is really no way to know what you can do so you
>just have to try to blast as much as you can" but that ignores the basic
>principles that have worked behind all internet services for a heck of a
>long time: just because you _can_ do something, doesn't mean you should do
>it if it isn't a friendly thing to do.
Hey, this isn't your father's Internet. Sendmail-like performance just
doesn't cut it. Poorly managed services/servers are likely to
drown. It may not be "nice" or "friendly", but my nice, friendly users
aren't infinitely patient: they want their mail delivered *now*.
>> No, but my tcpserver connection limit is sufficiently conservative
>> that it can almost always handle a full compliment of SMTP clients. If
>> it can't, so what? Chances are, a few too many SMTP connections when
>> the system is hosed are the least of my worries.
>
>"if I can't accept any mail from any other systems because one system
>decided to flood me with a huge amount just beacuse it thought it would be
>cool to schedule deliveries that way, then it isn't a big deal"
>
>Erm... I don't think that way.
I do.
>My claim is that qmail's behaviour ranges from extremely unfriendly to
>abusive. It is easy to end up with addresses sorted by hostname for a
>variety of reasons.
And very easy to prevent it, if it's a problem.
>It is a flaw in qmail that, if the address list is
>sorted in such a way, it takes ~twice as long, in my experience, to send
>out a large quantity of mail.
No complex system can be optimal at all times. In my experience, this
"flaw" just isn't a problem.
>> Of course not. What's your point? 256 is << 500000. If I've got 500000
>> different messages to deliver to a single site, I sure as hell want to
>> use as many simultaneous connections as I can. I assume the receiving
>
>You just said that, if qmail didn't have an arbitrary 256 simultaneous
>connection limit and if your machine could handle it, then you consider it
>perfectly acceptable to try to open 500000 simultaneous connections.
No I didn't. I said there's a big difference between 256 and
500000. Even with "well behaved" MTA's, a busy server today can expect
on the order of 256 connections from various hosts, so if my qmail
server should happen to get them all, the server shouldn't be
swamped. However, the busiest, most capable SMTP server in existence
couldn't handle 500000 connections, so I consider it unreasonable to
attempt that many. Ultimately, though, if my server accepts that many
connections, it's my responsibility to see that it can cope.
>> site will behave responsibly and only accept as many as it can handle.
>> If the don't, that's their problem.
>
>That sums up the problems with your attitude quite nicely, and the
>problems with far too many "qmail idealists" that are more concernend
>about proclaiming qmail to be the greatest thing on earth than about
>actually running functional and interoperable mail systems.
>
>It is obvious that you aren't interested in real life problems or actually
>using qmail in real life, but only on insisting that qmail is flawless, so
>there isn't too much point in continuing this discussion.
You're obviously wrong, though, because I use qmail daily, and have
for years, to deliver high volumes of mail to servers around the world
with concurrencyremote of up to 240. And--I've NEVER received a SINGLE
complaint from a remote site.
Put that in your "real life" pipe and smoke it.
-Dave
On Tue 1999-04-13 (09:56), Dave Sill wrote:
> I'm replying to several messages here (see References), but I'm not
> going to bother attributing each quote.
>
> >> >qmail will always be faster than sendmail [unless you send one message
> >> >to a large number of addresses on the same remote host].
> >>
> >> No, qmail will usually win here, too, because sendmail serializes.
> >> Sendmail only wins when the message is huge.
> >
> >Sendmail will win if you use multiple rcpt-to's.
>
> No, SMTP requires too many round trips per recipient.
What I had in mind was that with sendmail you can do:
HELO
MAIL FROM
RCPT TO: <address-1>
RCPT TO: <address-2>
....
RCPT TO: <address-n>
DATA
...
whereas with qmail, since it doesn't do multiple rcpts, you'd have to do:
for i = 1 to n
HELO
MAIL FROM
RCPT TO: <address-i>
DATA
...
QUIT
Remember that we're talking about sending one message to a large number of
addresses on the same remote host. In general qmail is faster, but I think in
this case any MTA that does multiple rcpt to's will be quicker.
> -Dave
- Keith
--
Keith Burdis - MSc (Com Sci) - Rhodes University, South Africa
Email : [EMAIL PROTECTED]
WWW : http://www.rucus.ru.ac.za/~keith/
IRC : Panthras JAPH
"Any technology sufficiently advanced is indistinguishable from a perl script"
Standard disclaimer.
---
Keith Burdis wrote/schrieb/scribsit:
> Remember that we're talking about sending one message to a large number of
> addresses on the same remote host. In general qmail is faster, but I think in
> this case any MTA that does multiple rcpt to's will be quicker.
AFAIK, sendmail has a limit of "RCPT TO"s it will send in one connection.
So in this situation a message will be split into several multi-rcpt
connections. Will these connections ocurr one at a time or simultaneously?
The former would impose a great speed penalty compared to qmail.
Anyway, the One True Answer on this topic is:
"profile, don't speculate".
Stefan
Keith Burdis <[EMAIL PROTECTED]> writes on 13 April 1999 at 21:39:09 +0000
> What I had in mind was that with sendmail you can do:
>
> HELO
> MAIL FROM
> RCPT TO: <address-1>
> RCPT TO: <address-2>
> ....
> RCPT TO: <address-n>
> DATA
> ...
>
> whereas with qmail, since it doesn't do multiple rcpts, you'd have to do:
>
> for i = 1 to n
> HELO
> MAIL FROM
> RCPT TO: <address-i>
> DATA
> ...
> QUIT
>
> Remember that we're talking about sending one message to a large number of
> addresses on the same remote host. In general qmail is faster, but I think in
> this case any MTA that does multiple rcpt to's will be quicker.
Yes, it does seem logical that the sendmail approach would be faster.
Have you experimented with this situation in the real world? People
who have report that in fact the qmail way is faster.
Performance analysis is important, but that analysis must not stop
with examining the algorithms; it must be carried out into the field,
to see what *really* happens in the real world. While there are cases
where qmail loses on delivery speed, they're far rarer than an
armchair analysis might suggest.
Or, to put it differently, in performance analysis, what "seems right"
is not always what *is* right.
--
David Dyer-Bennet [EMAIL PROTECTED]
http://www.ddb.com/~ddb (photos, sf) Minicon: http://www.mnstf.org/minicon
http://ouroboros.demesne.com/ The Ouroboros Bookworms
Join the 20th century before it's too late!
On Tue, 13 Apr 1999, Keith Burdis wrote:
> Remember that we're talking about sending one message to a large number of
> addresses on the same remote host. In general qmail is faster, but I think in
> this case any MTA that does multiple rcpt to's will be quicker.
if the effect latency of the connection is greater than the bandwidth
between the end systems multiple connections will complete faster than
multiple rcpts. In general this is true: the latency of my dialup
connection is a greater effect than the bandwidth on sending messages),
the latency between my ISP and the rest of the Internet is greater than
its bandwidth...
I think you should get information on latency for message from a machine
running a large distribution lst to see where it's spending its time
Richard
> I think you should get information on latency for message from a machine
> running a large distribution lst to see where it's spending its time
probably waiting for those slug domains which have either
* slow links
* slow dns
* are no longer in services
but, that is just a quick guess ;) note that all of these really are not
dependent upon whether you do one connection and pipe a bazillion messages
over it, or a bazillion connection of one message each. Imho,
i still think that the relatively random distribution of target
addresses for a server makes this relatively moot, unless
you are bulk-mailing 300,000 messages to AOL. however,
in that case, shouldn't the bulk mailer just talk SMTP and
save everyone the hassle?
-- craig
> What I had in mind was that with sendmail you can do:
>
> HELO
> MAIL FROM
> RCPT TO: <address-1>
> RCPT TO: <address-2>
> ....
> RCPT TO: <address-n>
> DATA
> ...
>
> whereas with qmail, since it doesn't do multiple rcpts, you'd have to do:
>
> for i = 1 to n
> HELO
> MAIL FROM
> RCPT TO: <address-i>
> DATA
> ...
> QUIT
>
> Remember that we're talking about sending one message to a large number of
> addresses on the same remote host. In general qmail is faster, but I think in
> this case any MTA that does multiple rcpt to's will be quicker.
>
> > -Dave
>
> - Keith
Dear Keith:
I think you got my point, thanks.
My experience on sendmail at this tpoic *is* quite good - sorry, I don't
have such experience on qmail yet.
My intuition tells me that the first way will be better, since it will send
only one copy of messages on network. If I have a 30KB+ message size, and
400K subscribers, the first method 'seems' to be much more efficient than
the second one.
However, this is what I 'think' - no real world experience on qmail yet.
I'll examine this behavior on qmail soon, the machine is still in the box,
I don't have time to open/assamble/install it these days.
Our newsletters have 'sponsors', and some information is time-intensive,
so my life is harder and harder.... :(
Anyway, thanks for any comment on this tpoic.
--
Regards
Silver CHEN
1999/4/14
Dear Sir:
I'm the one that causes 'a lot' discussion about the topic 'qmail speed'.
I've read through the whole mail-threads, and thanks for anyone that
give even a word here.
I HAVE to say that I like qmail, and I don't hate sendmail too -
that's not my point here. If anything other than these can solve my
problem, I'll choose it.
Someone said that qmail is weaker if I send many 'RCPT TO:' in one SMTP
transactions than sendmail. Well, I don't know the inside story, but I
do worry about that statement.
Usually the newsletter (or big mailing list) admin. will use 'bulk' mail
to distribute their message, i.e. one message with 'many' recipients.
And they will sort the recipient lists before sending to MTA too.
I think this flow is ok, but I don't know if this kind of behavior (one
message w/ many recipients, and sorted lists) is suitable for qmail?
Will qmail spwans a lot of processes for 'one' such SMTP transaction?
I choose 100 as the # of 'RCPT TO:' in one outgoing SMTP bulk, so there
will be more than 4000 such bulks sending to qmail-smtpd for a 400K
recipient list. What would happen there? I don't know now, but I'll
try recently.
If someone can comment on this topic, I'll be very appriciated, if
this place is not suitable, please just reply to me.
Thanks again and hope qmail will be better and better. Actually I
made another free-email (POP3) system using qmail+mysql, it's online
and I'll know what qmail can when I have, say, 500K accounts on it.
Regards.
--
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
| Shan-Ta CHEN E-Mail : [EMAIL PROTECTED] |
| Silver CHEN Tel(O) : +886-2-2773-9858-288 |
| �����F Tel(H) : +886-2-2914-1402 |
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~+
Silver CHEN <[EMAIL PROTECTED]> wrote:
>
> Someone said that qmail is weaker if I send many 'RCPT TO:' in one SMTP
> transactions than sendmail. Well, I don't know the inside story, but I
> do worry about that statement.
Don't worry. There are very rare situations in which sendmail can be
faster than qmail, but in 99+% of actual usage, qmail is *much*
faster than sendmail.
> Usually the newsletter (or big mailing list) admin. will use 'bulk' mail
> to distribute their message, i.e. one message with 'many' recipients.
> And they will sort the recipient lists before sending to MTA too.
It's probably better *not* to sort the lists with qmail because it
will pound on the receiving systems. If you randomize the recipients,
qmail's impact on receiving systems will be spread out.
> I think this flow is ok, but I don't know if this kind of behavior (one
> message w/ many recipients, and sorted lists) is suitable for qmail?
One message with many recipients is good. Sorted lists--at least,
sorted by domain or MX--is not *bad*, but may subject pooorly run
sites to more load than they can handle.
> Will qmail spwans a lot of processes for 'one' such SMTP transaction?
Yes, it'll spawn up to `concurrencyremote' processes, assuming
negligable local recipients. This is the big reason that qmail is
faster than sendmail: with sendmail, one process delivers to all
recipients, and only one connection is ever open to a remote
site. qmail, on the other hand, uses up to conncurrencyremote
(default: 20) processes, up to conncurrencyremote of which may be
connected to any given remote site at one time.
> I choose 100 as the # of 'RCPT TO:' in one outgoing SMTP bulk, so there
> will be more than 4000 such bulks sending to qmail-smtpd for a 400K
> recipient list. What would happen there? I don't know now, but I'll
> try recently.
That would put 4000 messages in the queue. If you increase the number
of recipients per message, qmail will be more efficient.
-Dave
On Tue, Apr 13, 1999 at 11:27:15AM -0400, Dave Sill wrote:
> Silver CHEN <[EMAIL PROTECTED]> wrote:
> >
> > Someone said that qmail is weaker if I send many 'RCPT TO:' in one SMTP
> > transactions than sendmail. Well, I don't know the inside story, but I
> > do worry about that statement.
>
> Don't worry. There are very rare situations in which sendmail can be
> faster than qmail, but in 99+% of actual usage, qmail is *much*
> faster than sendmail.
>
> > Usually the newsletter (or big mailing list) admin. will use 'bulk' mail
> > to distribute their message, i.e. one message with 'many' recipients.
> > And they will sort the recipient lists before sending to MTA too.
>
> It's probably better *not* to sort the lists with qmail because it
> will pound on the receiving systems. If you randomize the recipients,
> qmail's impact on receiving systems will be spread out.
>
> > I think this flow is ok, but I don't know if this kind of behavior (one
> > message w/ many recipients, and sorted lists) is suitable for qmail?
>
> One message with many recipients is good. Sorted lists--at least,
> sorted by domain or MX--is not *bad*, but may subject pooorly run
> sites to more load than they can handle.
>
> > Will qmail spwans a lot of processes for 'one' such SMTP transaction?
>
> Yes, it'll spawn up to `concurrencyremote' processes, assuming
> negligable local recipients. This is the big reason that qmail is
> faster than sendmail: with sendmail, one process delivers to all
> recipients, and only one connection is ever open to a remote
> site. qmail, on the other hand, uses up to conncurrencyremote
> (default: 20) processes, up to conncurrencyremote of which may be
> connected to any given remote site at one time.
Hmm very untrue in fact. Sendmail will under several circumstances [none of which I
will explain here but some of which are quite common] open several connections to the
same remote host, even for the same domain. The real problem is that sendmail is not
intelligent in this _at_ _all_. It'll just fire up another queue runner when it feels
like [when someone does sendmail -q (any user will do), or when mail comes in
(although the runner will then only attempt delivery of the mail that just came in),
or when given an ETRN (I give my fallback MX 7 ETRN commands when I dialin, of which
2 are for the same domain, which means sendmail always opens up 2 connections to my
qmail-smtpd :)]
hmmm some long sentence in there :)
Greetz, Peter
--
| 'He broke my heart, | Peter van Dijk |
I broke his neck' | [EMAIL PROTECTED] |
nognixz - As the sun | Hardbeat@ircnet - #cistron/#linux.nl |
| Hardbeat@undernet - #groningen/#kinkfm/#vdh |
Peter van Dijk <[EMAIL PROTECTED]> wrote:
>Dave Sill wrote:
>>
>> ...with sendmail, one process delivers to all
>> recipients, and only one connection is ever open to a remote
>> site. ...
>
>Hmm very untrue in fact. Sendmail will under several circumstances
>[none of which I will explain here but some of which are quite
>common] open several connections to the same remote host, even for
>the same domain.
You got me. Of course you're right. I meant to say that when
delivering a single message "immediately", i.e., not from the queue,
sendmail will only open one connection at a time.
-Dave
On Tue, Apr 13, 1999 at 01:23:17PM -0400, Dave Sill wrote:
> Peter van Dijk <[EMAIL PROTECTED]> wrote:
> >Dave Sill wrote:
> >>
> >> ...with sendmail, one process delivers to all
> >> recipients, and only one connection is ever open to a remote
> >> site. ...
> >
> >Hmm very untrue in fact. Sendmail will under several circumstances
> >[none of which I will explain here but some of which are quite
> >common] open several connections to the same remote host, even for
> >the same domain.
>
> You got me. Of course you're right. I meant to say that when
> delivering a single message "immediately", i.e., not from the queue,
> sendmail will only open one connection at a time.
No, that is one of the circumstances I'm referring to. If two incoming connections
each deliver a mail to the same remote destination, sendmail will happily fork() for
both and deliver them at the same time.
Greetz, Peter
--
| 'He broke my heart, | Peter van Dijk |
I broke his neck' | [EMAIL PROTECTED] |
nognixz - As the sun | Hardbeat@ircnet - #cistron/#linux.nl |
| Hardbeat@undernet - #groningen/#kinkfm/#vdh |
Peter van Dijk <[EMAIL PROTECTED]> wrote:
>On Tue, Apr 13, 1999 at 01:23:17PM -0400, Dave Sill wrote:
>>
>> You got me. Of course you're right. I meant to say that when
>> delivering a single message "immediately", i.e., not from the queue,
>> sendmail will only open one connection at a time.
>
>No, that is one of the circumstances I'm referring to. If two
>incoming connections each deliver a mail to the same remote
>destination, sendmail will happily fork() for both and deliver them
>at the same time.
That doesn't contradict what I said. By "single message", I was
excluding multiple messages.
Do you agree that a locally injected, immediate delivery mode message
with multiple recipients will be delivered serially by a single
sendmail process using one connection at a time? Of course, delivery
attempts that fail on the first try will be queued and available to
queue runners, but that's not my point.
-Dave
On Tue, Apr 13, 1999 at 03:36:20PM -0400, Dave Sill wrote:
> Peter van Dijk <[EMAIL PROTECTED]> wrote:
> >On Tue, Apr 13, 1999 at 01:23:17PM -0400, Dave Sill wrote:
> >>
> >> You got me. Of course you're right. I meant to say that when
> >> delivering a single message "immediately", i.e., not from the queue,
> >> sendmail will only open one connection at a time.
> >
> >No, that is one of the circumstances I'm referring to. If two
> >incoming connections each deliver a mail to the same remote
> >destination, sendmail will happily fork() for both and deliver them
> >at the same time.
>
> That doesn't contradict what I said. By "single message", I was
> excluding multiple messages.
Err ofcourse.. but opening multiple connections for one message would be slightly
ridiculous :)
> Do you agree that a locally injected, immediate delivery mode message
> with multiple recipients will be delivered serially by a single
> sendmail process using one connection at a time? Of course, delivery
> attempts that fail on the first try will be queued and available to
> queue runners, but that's not my point.
Yes I fully agree on that point. My point was that sendmail under certain
circumstances will have multiple connections to a single host, but no concurrency
limiting there, apart from sendmail pausing when the load gets too high.
Greetz, Peter
--
| 'He broke my heart, | Peter van Dijk |
I broke his neck' | [EMAIL PROTECTED] |
nognixz - As the sun | Hardbeat@ircnet - #cistron/#linux.nl |
| Hardbeat@undernet - #groningen/#kinkfm/#vdh |
>>Actually, if you are unfortunate enough to have a list of addresses sorted
>>by the right side of the @, qmail can be a big loser here. ...
>somedomain is poorly configured. Should qmail assume all sites are
>poorly configured? Should properly configured sites suffer because
>some sites are poorly run?
This is a topic I've been thinking about for a while. I want my MTA
to open as many connections to a remote site as the remote can handle.
If we're lucky, the remote is well configured and will reject
connections to tell us that it's busy, but as we all know, most MTAs
tend to accept them all and fall over.
Since qmail already keeps a retry interval for each remote IP it tries
to contact, how hard would it be also to keep some estimate of the
remote load, perhaps the time from accepting the connection to sending
the initial banner? Then it could limit the number of simultaneous
connections to slow hosts, for some definition of slow.
On a site with a lot of outgoing mail, I'd think this could improve
overall outgoing mail throughput, since it prevents qmail from doing a
DOS on itself by opening all of its outgoing connections to hosts that
are terminally slow. Instead, it favors deliveries to hosts that can
accept mail quickly and gets them out of the queue.
Yeah, none of this should be necessary, but there are a lot of
features in a robust MTA that wouldn't be necessary if the rest of the
world were better behaved.
--
John R. Levine, IECC, POB 727, Trumansburg NY 14886 +1 607 387 6869
[EMAIL PROTECTED], Village Trustee and Sewer Commissioner, http://iecc.com/johnl,
Member, Provisional board, Coalition Against Unsolicited Commercial E-mail
hello,
i have been tasked to talk to a linux conference here in the philippines
about qmail. now most of the guys in this technical conference are using
sendmail as their mail server and so am bound to be very scrutinized.
could anyone give me their reasons why they switched to qmail from
sendmail or any other mail server? anything convincing enough for most of
you would most likely be convincing for most other ppl not in the know :)
it's a 2 hr talk so i need some materials to talk about.
thanks !
-marlon
ps.
what can qmail do that sendmail can't do as of now and vice-versa?
On Tue, 13 Apr 1999, Marlon Anthony Abao wrote:
> could anyone give me their reasons why they switched to qmail
> from sendmail or any other mail server? anything convincing enough
> for most of you would most likely be convincing for most other ppl not
> in the know :)
One word: security.
When I was tasked with building mail relays for our new internet
connection, I wanted a program I wouldn't need to upgrade every month as a
new security exploit was found. That rules out sendmail.
As far as I'm concerned, sendmail has outlived its usefullness
because it can't shed itself of the baggage of 20 years.
--
gowen -- Greg Owen -- [EMAIL PROTECTED]
Greg Owen {gowen} <[EMAIL PROTECTED]> wrote:
>
>On Tue, 13 Apr 1999, Marlon Anthony Abao wrote:
>> could anyone give me their reasons why they switched to qmail
>> from sendmail or any other mail server? anything convincing enough
>> for most of you would most likely be convincing for most other ppl not
>> in the know :)
>
> One word: security.
>
> When I was tasked with building mail relays for our new internet
>connection, I wanted a program I wouldn't need to upgrade every month as a
>new security exploit was found. That rules out sendmail.
I couldn't agree more. That's why I switched to qmail. However, that
one word reason is unlikely to convince sendmail fans, who will
immediately counter that sendmail hasn't had a serious security
problem in months/years. You should be prepared to argue that that
doesn't mean sendmail is secure. I use the line "I'm not dead, but
that doesn't mean I'm immortal" to point out the fallacy of assuming
sendmail is secure (immortal) because it's not currently exploitable
(dead). Of course, the same can be said of qmail, so explain how qmail
was designed for security: the modularity, the mutually untrusting
components, etc., whereas sendmail was designed back when everyone on
the net knew everyone else, and everyone was well behaved and it has
an inherently insecure design.
-Dave
> I couldn't agree more. That's why I switched to qmail. However, that
> one word reason is unlikely to convince sendmail fans, who will
> immediately counter that sendmail hasn't had a serious security
> problem in months/years. You should be prepared to argue that that
> doesn't mean sendmail is secure.
Good point. I would, however, collect all the CERT reports that detail
holes in sendmail, and draw a graph of exploits found over time for Sendmail
and Qmail. Guess what, one line will look like a teenager with acne, one
will be bare...
Mind you, I still run sendmail internally, but a few more needs to mess
with the sendmail.cf macro configuration and I'll probably make the switch.
The biggest obstacle is that qmail is home-directory oriented, and I run a
sealed server.
> I use the line "I'm not dead, but
> that doesn't mean I'm immortal" to point out the fallacy of assuming
> sendmail is secure (immortal) because it's not currently exploitable
> (dead). Of course, the same can be said of qmail, so explain how qmail
> was designed for security: the modularity, the mutually untrusting
> components, etc., whereas sendmail was designed back when everyone on
> the net knew everyone else, and everyone was well behaved and it has
> an inherently insecure design.
Yes, "designed for security" says a lot of that for me.
I'd also point out the enormous amount of cruft. Sure, you can write a
tic tac toe processor with sendmail.cf format, but I'd prefer my mailer be
doing mail than playing games. That sort of complexity may have been
required way back when, but is it now? I don't think so. Another huge plus
of qmail is the simple configuration protocol (although the documentation
could be seriously improved; let's hope Russell's upcoming book does that).
--
gowen -- Greg Owen -- [EMAIL PROTECTED] -- [EMAIL PROTECTED]
Please note my new [EMAIL PROTECTED] address which will
become my default address in March, and which works now.
Does anyone have any experience (real world) with the speed/reliability of
IRIX NFS? We're looking at distributed ring of POP3/SMTP servers with a
single NFS server (right now) to hold all users' mail and directly accept
mail as the primary MX and a secondary MX to back it up. The POP3/SMTP
servers - two for now - wound check users mail via NFS and do no local
delivery.
-doug
Doug McClure wrote:
>
> Does anyone have any experience (real world) with the speed/reliability of
> IRIX NFS? We're looking at distributed ring of POP3/SMTP servers with a
> single NFS server (right now) to hold all users' mail and directly accept
> mail as the primary MX and a secondary MX to back it up. The POP3/SMTP
> servers - two for now - wound check users mail via NFS and do no local
> delivery.
One word comes to mind when I hear "NFS" and "mail" in the same
breath... Ick.
I admit I do not have any experience with qmail specifically in
this area, or even any with IRIX in this area (my IRIX
experience lies elsewhere), but the one time I was at a site
that delivered mail into an NFS mounted spool things were ugly.
Why do you want to use NFS? Why do you want to distribute
things?
The real question here is: What problem are you trying to
solve with this?
Whatever problem it is, I _doubt_ that NFS is the answer,
regardless of both the platform (e.g., IRIX) and the MTA (e.g.,
qmail).
Cheers,
David
--
David Lindes, KF6HFQ DaveLtd[tm] Enterprises
[EMAIL PROTECTED] http://www.daveltd.com/
>One word comes to mind when I hear "NFS" and "mail" in the same
>breath... Ick.
>
>I admit I do not have any experience with qmail specifically in
>this area, or even any with IRIX in this area (my IRIX
>experience lies elsewhere), but the one time I was at a site
>that delivered mail into an NFS mounted spool things were ugly.
>
>
>Why do you want to use NFS? Why do you want to distribute
>things?
>
>The real question here is: What problem are you trying to
>solve with this?
>
>Whatever problem it is, I _doubt_ that NFS is the answer,
>regardless of both the platform (e.g., IRIX) and the MTA (e.g.,
>qmail).
My experience (and others on this list) differs. Maildir on a reliable NFS
system is an excellent way of sharing a filestore across multiple front end
systems.
A common model is multiple POP servers (Maildir ones that is) using a NetApp
box. It works exceedingly well.
I agree that NFS is *not* well suited to V7 mailboxes and the associated
locking.
So, returning to the original question. If IRIX provides a quality NFS
implementation and you plan to stash incoming email in Maildir format, then
it should work quite well.
Regards.
Mark Delany wrote:
>
> My experience (and others on this list) differs. Maildir on a reliable NFS
> system is an excellent way of sharing a filestore across multiple front end
> systems.
Hmm... interesting.
> I agree that NFS is *not* well suited to V7 mailboxes and the associated
> locking.
Ahh. Yes, that would make the differenc. :-)
> So, returning to the original question. If IRIX provides a quality NFS
> implementation and you plan to stash incoming email in Maildir format, then
> it should work quite well.
My experience with the IRIX NFS client would tend to say that it
is a reasonably good one.
Cheers,
David
--
David Lindes, KF6HFQ DaveLtd[tm] Enterprises
[EMAIL PROTECTED] http://www.daveltd.com/
Dave,
we have been deploying a very large mail service with the home directories
are mounted on NFS. we currently have 5 SMTP/POP3 boxes, each with thier
own queues accepting connections via a round-robin dns.
this boxes are the front-ends for another 5 Data servers which just serves
as NFS servers. all auth is done via NIS.
this setup is behind a firewall which blocks out anything incoming except
smtp/pop3, http/ftp and radius packets.
we are running all these machines on Linux 2.2.x kernels on the x86
architechture. and the only time we need to restart any machine if for
kernel upgrades :)
-marlon
At 11:12 PM 4/13/99 -0700, you wrote:
>Doug McClure wrote:
>One word comes to mind when I hear "NFS" and "mail" in the same
>breath... Ick.
>
>I admit I do not have any experience with qmail specifically in
>this area, or even any with IRIX in this area (my IRIX
>experience lies elsewhere), but the one time I was at a site
>that delivered mail into an NFS mounted spool things were ugly.
>
>
>Why do you want to use NFS? Why do you want to distribute
>things?
>
>The real question here is: What problem are you trying to
>solve with this?
>
>Whatever problem it is, I _doubt_ that NFS is the answer,
>regardless of both the platform (e.g., IRIX) and the MTA (e.g.,
>qmail).
>
>
>Cheers,
>
> David
>
>--
>David Lindes, KF6HFQ DaveLtd[tm] Enterprises
>[EMAIL PROTECTED] http://www.daveltd.com/
>
>
I'm trying to setup qmail to have 2 sepearate domains and 2 seperate
usernames. for example
[EMAIL PROTECTED] and [EMAIL PROTECTED] are 2 seperate boxes,
but want to be able to host them on the same machine, but keep each billybob
seperate. is this possible with qmail?
Gary Stewart
[EMAIL PROTECTED]
Hi, I like users to be able to access and manage their email via a browser.
Does anyone know of a web-based interface to Qmail or other similar packages
for Linux that is freeware?
Thanks,
Matthew Bora Kaing
On Tue, Apr 13, 1999 at 11:11:50AM -0700, Matthew Kaing wrote:
> Hi, I like users to be able to access and manage their email via a browser.
> Does anyone know of a web-based interface to Qmail or other similar packages
> for Linux that is freeware?
web.horde.org/imp/, if I'm not mistaken.
Greetz, Peter
--
| 'He broke my heart, | Peter van Dijk |
I broke his neck' | [EMAIL PROTECTED] |
nognixz - As the sun | Hardbeat@ircnet - #cistron/#linux.nl |
| Hardbeat@undernet - #groningen/#kinkfm/#vdh |
At 11:11 am -0700 13/4/99,the wonderful Matthew Kaing wrote:
>Hi, I like users to be able to access and manage their email via a browser.
>Does anyone know of a web-based interface to Qmail or other similar packages
>for Linux that is freeware?
www.endymion.com/products/mailman
--
peter at gradwell dot com; http://www.gradwell.com/
gradwell dot com Ltd. Enabling the internet you don't see.
Matthew Kaing writes:
> Hi, I like users to be able to access and manage their email via a browser.
> Does anyone know of a web-based interface to Qmail or other similar packages
> for Linux that is freeware?
http://www.inter7.com/sqwebmail/
Only for Maildir mailboxes.
--
Sam
http://www.focalmail.com/
--
Y2K - We're all gonna die.
Hi all,
I'm a bit of a newbie to qmail, so my apologies in advance if
this is a FAQ or if I don't explain things correctly, but here
goes:
I've got a setup on my ISP's machine using qmail where they are
hosting several virtualdomains for me, and I am setting up
mailing lists within some of those domains with ezmlm.
The problem I'm seeing is that the envelope from header on
messages going out from ezmlm to me or other people on the same
ISP are getting re-written (or written in the first place by
ezmlm, I'm not sure which) such that instead of having
listname-return-id-user=domain, they have
listname-return-id-qmaildeliverexpansion=domain
For example, when sending mail to [EMAIL PROTECTED], which
[EMAIL PROTECTED] is a subscriber of, instead of having:
[EMAIL PROTECTED]
I get:
[EMAIL PROTECTED]
Because the virtualdomains entry (in /var/qmail/control) for
daveltd.com looks like this:
daveltd.com:lindes-daveltd.d/com/q
This may not be a real problem, since mail to
[EMAIL PROTECTED] isn't bouncing (and therefore mail wouldn't
get sent to the above envelope address), but it worries me a
bit...
BTW, the envelope header for users in domains that aren't in the
virtualdomains file on this machine don't seem to be affected,
and just show up as listname-return-id-user=domain, as I believe
they should.
Is this a bug? A configuration problem on either my side, or
something in qmail on the machine as a whole? Any easy way for
me to figure out at what level (ezmlm stuff or qmail-inject) the
address is getting written this way?
Thanks for any help.
Cheers,
David
--
David Lindes, KF6HFQ DaveLtd[tm] Enterprises
[EMAIL PROTECTED] http://www.daveltd.com/
Hi. I have virtual domains on our server. One of which is nabla.com.br.
We use rcpthosts to block spammers and tcpserver to allow pop. Our main
domain is pcshop.com.br, which works fine. My virtualdomains reads like
this:
---------------------------------------------------------------------------
[EMAIL PROTECTED]:pronto
[EMAIL PROTECTED]:pronto
[EMAIL PROTECTED]:pronto
[EMAIL PROTECTED]:pronto
[EMAIL PROTECTED]:pronto
[EMAIL PROTECTED]:ebidsp
[EMAIL PROTECTED]:ebidsp
[EMAIL PROTECTED]:ebidsp
[EMAIL PROTECTED]:ebidsp
[EMAIL PROTECTED]:ebidsp
[EMAIL PROTECTED]:ebidsp
[EMAIL PROTECTED]:nabla
[EMAIL PROTECTED]:nabla
[EMAIL PROTECTED]:nabla
[EMAIL PROTECTED]:nabla
[EMAIL PROTECTED]:nabla
[EMAIL PROTECTED]:nabla
---------------------------------------------------------------------------
But when I send from an account in other server (a webmail free service)
to [EMAIL PROTECTED], I get the following bounce:
---------------------------------------------------------------------------
The original message was received at Tue, 13 Apr 1999 18:45:46 -0300
(EST)
from [200.246.70.60]
----- The following addresses had permanent fatal errors -----
<[EMAIL PROTECTED]>
----- Transcript of session follows -----
.... while talking to pcs001.pcshop.com.br.:
>>> RCPT To:<[EMAIL PROTECTED]>
<<< 553 sorry, that domain isn't in my list of allowed rcpthosts
(#5.7.1)
550 <[EMAIL PROTECTED]>... User unknown
---------------------------------------------------------------------------
I even tried to put nabla.com.br in the locals file, tried to touch
~alias/.qmail-tecnologia to no avail. Oh, and .qmail-nabla-tecnologia is
like this:
---------------------------------------------------------------------------
&[EMAIL PROTECTED]
&[EMAIL PROTECTED]
---------------------------------------------------------------------------
What can be happening?
--
___THE___ "Commercial OS vendors are, at the moment, all closed
\ \ / / economies, and doomed to fall in their competition with
\ V / open economies just as communism eventually fell."
\ / -- H. Reiser, Unix OS developer
/ \ _____________________________________________________
/ ^ \ | Juan Carlos Castro y Castro - [EMAIL PROTECTED] |
/ / \ \ | Diretor de Inform�tica e Eventos Sobrenaturais da |
~~~ ~~~ | E-RACE CORPORATION |
RACER -----------------------------------------------------
>But when I send from an account in other server (a webmail free service)
>to [EMAIL PROTECTED], I get the following bounce:
>
>---------------------------------------------------------------------------
>The original message was received at Tue, 13 Apr 1999 18:45:46 -0300
>(EST)
> from [200.246.70.60]
>
> ----- The following addresses had permanent fatal errors -----
> <[EMAIL PROTECTED]>
>
> ----- Transcript of session follows -----
> .... while talking to pcs001.pcshop.com.br.:
> >>> RCPT To:<[EMAIL PROTECTED]>
> <<< 553 sorry, that domain isn't in my list of allowed rcpthosts
>(#5.7.1)
> 550 <[EMAIL PROTECTED]>... User unknown
>---------------------------------------------------------------------------
>
>I even tried to put nabla.com.br in the locals file, tried to touch
>~alias/.qmail-tecnologia to no avail. Oh, and .qmail-nabla-tecnologia is
>like this:
Right. How about putting nabla.com.br in /var/qmail/control/rcpthosts?
Regards.
Mark Delany wrote:
>
> Right. How about putting nabla.com.br in /var/qmail/control/rcpthosts?
I did!!! That's why it's weird! :-O
--
___THE___ "Commercial OS vendors are, at the moment, all closed
\ \ / / economies, and doomed to fall in their competition with
\ V / open economies just as communism eventually fell."
\ / -- H. Reiser, Unix OS developer
/ \ _____________________________________________________
/ ^ \ | Juan Carlos Castro y Castro - [EMAIL PROTECTED] |
/ / \ \ | Diretor de Inform�tica e Eventos Sobrenaturais da |
~~~ ~~~ | E-RACE CORPORATION |
RACER -----------------------------------------------------
Something occurred to me:
Juan Carlos Castro y Castro wrote:
>
> But when I send from an account in other server (a webmail free service)
> to [EMAIL PROTECTED], I get the following bounce:
>
> >>> RCPT To:<[EMAIL PROTECTED]>
> <<< 553 sorry, that domain isn't in my list of allowed rcpthosts
> (#5.7.1)
> 550 <[EMAIL PROTECTED]>... User unknown
Could it be the sending server that malformed the RCPT line in the SMTP
dialog? Maybe qmail-smtpd got confused and isn't parsing it right?
--
___THE___ "Commercial OS vendors are, at the moment, all closed
\ \ / / economies, and doomed to fall in their competition with
\ V / open economies just as communism eventually fell."
\ / -- H. Reiser, Unix OS developer
/ \ _____________________________________________________
/ ^ \ | Juan Carlos Castro y Castro - [EMAIL PROTECTED] |
/ / \ \ | Diretor de Inform�tica e Eventos Sobrenaturais da |
~~~ ~~~ | E-RACE CORPORATION |
RACER -----------------------------------------------------
At 07:37 PM Tuesday 4/13/99, Juan Carlos Castro y Castro wrote:
>Something occurred to me:
>
>Juan Carlos Castro y Castro wrote:
>>
>> But when I send from an account in other server (a webmail free service)
>> to [EMAIL PROTECTED], I get the following bounce:
>>
>> >>> RCPT To:<[EMAIL PROTECTED]>
>> <<< 553 sorry, that domain isn't in my list of allowed rcpthosts
>> (#5.7.1)
>> 550 <[EMAIL PROTECTED]>... User unknown
>
>Could it be the sending server that malformed the RCPT line in the SMTP
>dialog? Maybe qmail-smtpd got confused and isn't parsing it right?
Possible I guess. I just tried it by hand and the MX for nabla accepted that
domain just fine?
Regards.
Dave Sill wrote (on Apr 12, 1999):
> [EMAIL PROTECTED] wrote:
>
> >What are the advantages/disadvantages of cyclog over syslog?
[...]
> Disadvantages: only logs messages sent to stdout, only logs messages
> from local system, doesn't chunk logs by day/week/etc--only by size,
> dates/times aren't human-readable without "tailocal",
This last disadvantage can be perhaps better served, for those of
emacs persuasion, by the hack below.
--
Arnaldo Mandel
Computer Science Dept.
Universidade de S\~{a}o Paulo, Brazil
[EMAIL PROTECTED]
;;; tailocal.el - handles TAI time-stamps
;; Time-stamp: <1999/04/13 19:15:54 benavuya.ime.usp.br [benavuya.ime.usp.br] am>
;;; Copyright (C) 1999 Arnaldo Mandel
;;
;; Author: Arnaldo Mandel ([EMAIL PROTECTED])
;; Version: 0.1
;;
;; This program is free software; you can redistribute it and/or modify
;; it under the terms of the GNU General Public License as published by
;; the Free Software Foundation; either version 2 of the License, or
;; (at your option) any later version.
;;
;; This program is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
;; GNU General Public License for more details.
;; This substitutes TAI time-stamps by readable timestamps.
;; Mostly usable for reading djb-style logfiles.
;;
;; If those are useful for you, you would probabley bind
;; the interactive functions to keys. Suggestion:
;; (eval-after-load "dired"
;; '(define-key dired-mode-map "\C-t" 'dired-decode-tai-filename))
;; (define-key text-mode-map "\C-xt" 'tailocal)
(defvar time-stamp-format "[%Y/%m/%d %T]")
(defconst tai-stamp "\\(^\\| \\)\\(\\([0-9]+\\)\\.[0-9]+\\) ")
(defun tai-to-local (num)
(format-time-string time-stamp-format
(cons (+ 12288 (ash num -16)) (mod num 65536))))
(defun match-tai ()
(if (re-search-forward tai-stamp (point-max) t)
(let ((num (string-to-number (match-string 3))))
(replace-match
(tai-to-local num) t t nil 2)
(backward-char)
t)))
(defun dired-decode-tai-filename ()
"Decodes the current filename as cyclog's @TAI."
(interactive)
(dired-move-to-filename)
(momentary-string-display
(tai-to-local
(string-to-number (buffer-substring (1+ (point)) (+ 15 (point))))) (point)))
; Alternative implementation. Maybe there should be a var deciding among these two
;(defun dired-decode-tai-filename ()
; "Decodes the current filename as cyclog's @TAI."
; (interactive)
; (message (tai-to-local (string-to-number (substring (dired-get-filename) -12)))))
(defun tailocal (from to)
"Converts tai time-stamps on REGION to human-readable ones."
(interactive "r")
(let ((vezes 0)
(matches 0)
(q 0)
(m (copy-marker to))
(bm (buffer-modified-p)))
(save-excursion
(message "Tailocalizing:")
;; Estimate number of tai-stamps, for feedback
(goto-char to)
(forward-line -1)
(setq q (point))
(forward-line -10)
(save-match-data
(while (re-search-forward tai-stamp q t)
(setq matches (1+ matches))))
(setq matches (/ (* matches (count-lines from to)) 10)
q (1+ (/ matches 100)))
;; Now do the work
(goto-char from)
(while (and (< (point) m)
(match-tai))
(setq vezes (1+ vezes))
(if (= (% vezes q) 0)
(message "Tailocalizing: %d%% done." (/ (* 100 vezes) matches)))))
(message "")
(set-buffer-modified-p bm)))
Hi:
I have installed & configured qmail on my
freebsd machine running FreeBSD 2.2-960130-SNAP.
I am running into some strange qmail behavior.
I have configured qmail under tcpserver.
Queues are permanently stuck and messages keep
accruing in the queue. When the machine is rebooted,
tcpserver starts, apparently qmail also does, but
qmail dies.
During reboot or any time I restart qmail, I get a
handful of messages that are delivered but are never
cleared from the queue. In other words, if I reboot
again, I get the same set of messages again from the
queue. Also, kill -ALRM or running tcp_ok seems to
make absolutely no sense. When I run qmail-qstat, all
messages are shown as being in queue, with 0 messages
not yet preprocessed (value 0 is true all the time).
When I restart qmail manually, I do see all the processes such as
qmail-send, rspawm, lspawn, splogger,
etc. As explained earlier, qmail does not stay up
after a reboot but tcpserver does.
Any ideas, tips will be appreciated. I also tried
doing a make setup check without any success.
Wonder if this has anything to do with my old version
of FreeBSD and or locking. I have also run the
qmail-sanity perl script, it does not show any errors.
Thanks in advance, Dinesh.
_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com
>I am running into some strange qmail behavior.
>I have configured qmail under tcpserver.
>
>Queues are permanently stuck and messages keep
>accruing in the queue. When the machine is rebooted,
>tcpserver starts, apparently qmail also does, but
>qmail dies.
Apparently? You don't know? What interpretation have you made of the logs?
>During reboot or any time I restart qmail, I get a
>handful of messages that are delivered but are never
>cleared from the queue. In other words, if I reboot
>again, I get the same set of messages again from the
>queue. Also, kill -ALRM or running tcp_ok seems to
>make absolutely no sense. When I run qmail-qstat, all
>messages are shown as being in queue, with 0 messages
>not yet preprocessed (value 0 is true all the time).
>
>When I restart qmail manually, I do see all the processes such as
>qmail-send, rspawm, lspawn, splogger,
>etc. As explained earlier, qmail does not stay up
>after a reboot but tcpserver does.
>Any ideas, tips will be appreciated. I also tried
>doing a make setup check without any success.
Show us what it generated. There is no point in trying to delivery mail if
"make check" fails.
If you mean that it didn't fail, then show us some logs of the delivery
attempts. Show us some ps output of all qmail processes. Show us a process
trace of qmail-send.
In other words, we don't want a paraphrase of your problem, show us some
real data if you wish to have some hope of a useful reply.
Regards.
Hi Dirk et al,
Hmm, after reading this thread, I see that I have a thought on
how to help with this that doesn't seem to exactly be stated
here... (though some similar things have been)
I venture this suggestion with some trepidation, being that a)
I'm a newbie to qmail, and b) I'm a newbie to this list, and
also because I see some reference to "the great fork/exec"
wars. But I suspect there's still at least _some_ validity to
my observation (which is made based on my experience with
non-qmail things), so I'll present it anyway...
Note: The core of my observation is based on a suspicion that
it's not really qmail's fault at all (well, not entirely,
anyway) that this is slow, but rather the enclosing loop.
If you're of the belief that the above premise is false, and
not interested in experimenting with this, then stop reading
now. :-)
[EMAIL PROTECTED] wrote (a few words more than) the following:
>
> A client of ours is delivering a newsletter to 230,000
> people which we are feeding into the queue like this:
>
> #! /bin/sh
>
> for address in `cat list`
> do
>
> echo -ne "[EMAIL PROTECTED]\000T$address\000\000" >/tmp/address
> sed s/xxxx/$address/g /tmp/message | /var/qmail/bin/qmail-queue 1< /tmp/address
>
> echo $address >>log
> done
>
> Anybody know a better/faster way?
Well, it seems to me that you're doing a lot of fork/exec pairs
(for sed) and open/close pairs (your /tmp/address file) that
aren't quite necessary...
In my experience, doing a shell loop with fork/exec pairs in the
middle of it is _hugely_ less efficient than, say, a perl loop
with no fork/exec that achieves the same effect. The solution I
present, though, doesn't get rid of _all_ fork/exec's, so the
improvement will certainly be less dramatic... But I suspect it
will still be present.
Note also that my solution assumes you have RAM to burn on
pre-reading your recipient list. If that's not the case, you'll
particularly want to try changing that portion of the below.
I consider the following to be pseudo-code (and it's
pseudo-perl). Most of it is actually probably genuine perl
code (i.e. it might actually work), but I have left parts out
(I'm not terribly practiced at doing writing to multiple
file-descriptors in a pipeline, for example, so I'll leave that
as just comments giving pointers to things you might use to do
this), I don't have error checking everywhere I should, and I
have not done any checks on what's there... Most of the places
where I know I've left something out I have flagged with
'XXX'. You'll want to change those pieces.
I've not tested any of this, etc. etc. etc. (use StdDisclaimer; :-)
... but, here goes:
#### begin
$parallelism = 10; # you'll want to experiment with different
# values here. Don't set it to less than 1,
# but any positive integer should be doable.
# (within reason, of course. ;-)
# Note: a value of 1 most closely resembles
# your stated solution.
open(RECIPS, "list") ||
die("Couldn't open recipient list: $!\n");
@recipients = <RECIPS>; # slurp the whole file. Note that this
# assumes one recip per line.
close(RECIPS);
chomp(@recipients); # strip newlines
undef($/); # go into file-at-a-time read mode
open(MESSAGE, "/tmp/message") ||
die("Couldn't open /tmp/message: $!\n");
$message_master = <MESSAGE>; # slurp the message into memory
close(MESSAGE);
$plid = 0; # parallelism id for the main parent
# note that the below loop doesn't execute if $parallelism is 1.
# That is the correct behavior. We want $parallelism - 1 children.
for($i = 1; $i < $parallelism; $i++)
{
if(($pid = fork()) == 0) # child
{
$plid = $i; # set the plid for this child
}
# parent doesn't need to do anything yet.
# XXX should really check for failures too, by checking for
# negative $pid.
}
# we no longer know, or care, if we are the master or one of the
# children from the loop above... We have our plid, which is what
# matters to us now.
# now loop through all the recipients that this plid owns (this is the
# main loop that actually does the work):
for($i = $plid; $i < scalar(@recipients); $i += $parallelism)
{
# copy the data for the message. Need to copy it so we don't
# break the master copy, which we'll need in the next iteration.
$message_copy = $message_master;
# change all instances of 'xxxx' to the current recipient:
$message_copy = s/xxxx/$recipients[$i]/sg;
# XXX do some pipe stuff. I'll assume we create "FILDES0" and
# "FILDES1" for the master to write from.
if(($pid = fork()) == 0) # child
{
# XXX do some magic to make FILDES0 and FILDES1 actually be
# fd's 0 and one for the child. Ala dup(3C), but I think in
# perl it involves open() with ">&" type args... See the
# perlipc man page for more.
exec("/var/qmail/bin/qmail-queue") ||
die("Oops, exec of qmail-queue failed for recipient " .
"'$recipients[$i]'.\n");
}
elsif($pid > 0) # parent
{
print(FILDES0 $message_copy); # XXX error checking?
print(FILDES1 "[EMAIL PROTECTED]\000T$address\000\000");
# XXX again... errors?
# XXX wait for child to finish, close file descriptors, etc.
}
# XXX check for (and deal with) fork failure here, too.
}
#### end
So, it's about 8 times as many lines of code, but it cut's out
the whole sed thing, and gives you customizable parallelism,
which I suspect _might_ be quite helpful. (Though that would
depend a lot on the system... Having 10 (or whatever) of these
things might slow the system down enough to negate the benefits
of parallelism, I dunno.) However, if my initial premise is
correct, this could make a world of difference (for the better).
Best of luck!
Cheers,
David
--
David Lindes, KF6HFQ DaveLtd[tm] Enterprises
[EMAIL PROTECTED] http://www.daveltd.com/
Hello all,
When a pop user logs in to check mail, they send their user password in clear
text over the network. So, a pop user account could be comprimised, and is
therefore unsecure. On a mail server I administer, I set all of the qmail user
accounts shell to be /bin/false which disallows a direct login by the user. This
is fine with me since none of my email accounts will every log in.
This seems secure, but is it enough? Is there more that one can do to secure pop
accounts?
--
Joseph R. Junkin Datafree Corporation
[EMAIL PROTECTED] http://www.datacrawler.com
On Wed, Apr 14, 1999 at 12:08:05AM -0400, Joe Junkin wrote:
> Hello all,
> When a pop user logs in to check mail, they send their user password in clear
> text over the network. So, a pop user account could be comprimised, and is
> therefore unsecure. On a mail server I administer, I set all of the qmail user
> accounts shell to be /bin/false which disallows a direct login by the user. This
> is fine with me since none of my email accounts will every log in.
>
> This seems secure, but is it enough? Is there more that one can do to secure pop
> accounts?
If the accounts are _only_ for email, you should consider a vpop solution, putting
all mailboxes under 1 UID.
IMnsHO, no pop-only account should be in your /etc/passwd at _any_ time.
Greetz, Peter
--
| 'He broke my heart, | Peter van Dijk |
I broke his neck' | [EMAIL PROTECTED] |
nognixz - As the sun | Hardbeat@ircnet - #cistron/#linux.nl |
| Hardbeat@undernet - #groningen/#kinkfm/#vdh |
