qmail Digest 9 Sep 2001 10:00:01 -0000 Issue 1482
Topics (messages 69186 through 69196):
QMQP may make qmail-send unable to send and bounce your mail (was: QMQP may eat your
mail and its bounces)
69186 by: Matthias Andree
Re: Reliability? What's that?
69187 by: Matthias Andree
Re: qmail-analog and tai64 logs
69188 by: Alex Stevens
69189 by: Eric Paynter
Re: duplicate email throttle
69190 by: Jeremy Hansen
69191 by: Jeremy Hansen
69193 by: Adrian Ho
Suggestion to fix the "bounce doesn't get out either" issue (was: QMQP may make
qmail-send unable to send and bounce your mail)
69192 by: Matthias Andree
I'm blocking ORBZ scans
69194 by: Russell Nelson
553 sorry, that domain isn't in my list of allowed rcpthosts
69195 by: Charles Widdis
69196 by: Adrian Ho
Administrivia:
To unsubscribe from the digest, e-mail:
[EMAIL PROTECTED]
To subscribe to the digest, e-mail:
[EMAIL PROTECTED]
To bug my human owner, e-mail:
[EMAIL PROTECTED]
To post to the list, e-mail:
[EMAIL PROTECTED]
----------------------------------------------------------------------
On Sat, 08 Sep 2001, D. J. Bernstein wrote:
> > qmail-send creates bounces that are not properly-formatted in that they
> > don't pass SMTP.
>
> False. qmail-send creates a bounce message in the extended format only
> if the original message was in the extended format.
So it does, and my claim is not "false". qmail cannot reliably bounce
"extended format" (whatever that is), that's a fact you acknowledged.
That IS a reliability problem no matter what YOU call it.
If (favourite contestant) did this, you'd put up a web page about
(favourite contestant) mail disasters, showing that (favourite
contestant) ate mail. See all that outdated Postfix discussion which is
obsolete since 1998-12-25 you just keep up because you feel as the
user's lawyer without actually having a mandate.
However, now I know better and will tell all people not to run
qmail-qmqpd to prevent their users shooting themselves in their feet.
The SMTP latency issues you point out ("1,000 recipient mail across 28.8
kbps modem link") are non-existant with ESMTP PIPELINING.
Bad thing qmail-remote can't do it, but that's your own business.
To the rest of the list, the patch I sent is incomplete and just fixes
the "missing terminated line" case. I didn't expect qmail to be that
blatantly ignorant of existing restrictions. Watch your mail logs
closely (from cron or functional equivalents) for discarded mail if you
run qmail-qmqpd.
--
Matthias Andree
Outlook (Express) users: press Ctrl+F3 for the full source code of this post.
begin dont_click_this_virus.exe
end
On Sat, 08 Sep 2001, D. J. Bernstein wrote:
> Matthias Andree writes:
> > qmail needs 30,000 write operations for these 1,000 mails, Postfix
> > needs 6,250.
>
> Gee. When you say ``Postfix,'' are you referring to the same MTA that
> incorrectly skips some essential fsyncs, because the MTA author didn't
> understand that write-fchmod-fsync often writes the inode first?
No, you must mean a different Postfix. My Postfix correctly saves fsyncs
because it does not rely on file system write order even though it may
look so to the casual observer.
Seriously, I had the same fear, but if you look at the parts that read
the queue, the +x is necessary, but not sufficient to them. Check
src/global/mail_open_ok.c for details.
Now stop lying to the world.
Stop defaming authors that checked their facts.
Check _your_ facts.
Read the 1999 "Soft Updates: A Technique for Eliminating Most
Synchronous Writes in the Fast Filesystem" Usenix paper by Marshall Kirk
McKusick and Gergory R. Ganger.
Then, review the Postfix queue format.[1]
> You know, I can save _tons_ of writes by storing mail on a RAM disk! Why
> am I bothering to implement a fast journaling filesystem for qmail 2,
> when I could simply pretend that the power never goes out?
False. You can't save the writes, you just get rid of spinning-disk
latency.
Why you are bothering? Tell us, we can only speculate.
[1] Postfix writes records into its queue file which have length and
type fields, queue files with truncated records or with missing end
of file record are rejected, and will not have been acknowledged to
the client. Acknowledgements are only issued after fsync() has
completed, and at that time, inode AND file are on disk.
--
Matthias Andree
Outlook (Express) users: press Ctrl+F3 for the full source code of this post.
begin dont_click_this_virus.exe
end
On Fri, Sep 07, 2001 at 11:40:46PM -0700, Eric Paynter wrote:
> I'm using "multilog t ... " to log the events of qmail-send. I just
> downloaded qmail-analog and the docs indicate that it wants the timestamps in
> the format "901967408.113926" instead of my "@400000003b9099e60abb1bc4". Is
> there an easy way to convert between these two formats?
This was asked within the past few days. You want qlogtools. Search google. The
specific tool is tai64n2tai, which converts to the older TAI format.
-Alex
--
__________________
[EMAIL PROTECTED]
------------------
"Yeah, I'm drunk all right, and you're crazy. Tomorrow I'll
be sober, but you'll still be crazy for the rest of your life."
-W.C. Fields
thanks for your replies ... I've found it.
-Eric
On September 8, 2001 08:28 am, Alex Stevens wrote:
> On Fri, Sep 07, 2001 at 11:40:46PM -0700, Eric Paynter wrote:
> > I'm using "multilog t ... " to log the events of qmail-send. I just
> > downloaded qmail-analog and the docs indicate that it wants the
> > timestamps in the format "901967408.113926" instead of my
> > "@400000003b9099e60abb1bc4". Is there an easy way to convert between
> > these two formats?
>
> This was asked within the past few days. You want qlogtools. Search google.
> The specific tool is tai64n2tai, which converts to the older TAI format.
>
> -Alex
--
arctic bears - the internet - your way.
email hosting from US$8/month, domains from US$19/year.
http://www.arcticbears.com
It doesn't seem that using reformail is going to work for my case.
Reformail looks for Message-ID's that are duplicates. In my case, the
Message-ID are not duplicates, just the subject and the body are the same,
so I need something that works on that level. The messages I'm trying to
limit are status messages generated by machines I'm monitoring...sometimes
the machines decide to send out 3000 or so messages in several minutes.
Each has a unique Message-ID because they are individually generates.
I'm going to look at the other duplicate eliminators on qmail.org, since
it seems they do some type of MD5sum on the message itself.
Any other ideas are welcome, or, perhaps I'm misunderstanding how
reformail works.
-jeremy
On Sat, 8 Sep 2001, Adrian Ho wrote:
> On Sat, Sep 08, 2001 at 03:30:55PM +0800, Adrian Ho wrote:
> > <http://www.qmail.org/>, search for "duplicate", install and configure
> > accordingly for those users who don't want dups.
>
> After some more thought, maildrop has a utility (reformail) that makes
> the above unnecessary. For each user X who wants dups eliminated,
> ~X/.qmail becomes:
>
> | ! reformail -D 50000 .msg-ids || exit 99
> &X-delivery
>
> and ~X/.qmail-delivery contains:
>
> ./Mail/inbox/
>
> or however you want the non-dup mails handled.
>
> Read the reformail and dot-qmail man pages to understand how the above
> pipeline works.
>
>
--
In a gathering of two or more people, when a lighted cigarette is
placed in an ashtray, the smoke will waft into the face of the non-smoker.
To expand on this a bit, I need something that says:
message a with the same body and subject is coming in at 1 message per
second, instead of forwarding all this mail that's coming in at this rate,
send a single message. So I need something that keeps time as well it
seems.
-jeremy
On Sat, 8 Sep 2001, Jeremy Hansen wrote:
>
> It doesn't seem that using reformail is going to work for my case.
> Reformail looks for Message-ID's that are duplicates. In my case, the
> Message-ID are not duplicates, just the subject and the body are the same,
> so I need something that works on that level. The messages I'm trying to
> limit are status messages generated by machines I'm monitoring...sometimes
> the machines decide to send out 3000 or so messages in several minutes.
> Each has a unique Message-ID because they are individually generates.
>
> I'm going to look at the other duplicate eliminators on qmail.org, since
> it seems they do some type of MD5sum on the message itself.
>
> Any other ideas are welcome, or, perhaps I'm misunderstanding how
> reformail works.
>
> -jeremy
>
> On Sat, 8 Sep 2001, Adrian Ho wrote:
>
> > On Sat, Sep 08, 2001 at 03:30:55PM +0800, Adrian Ho wrote:
> > > <http://www.qmail.org/>, search for "duplicate", install and configure
> > > accordingly for those users who don't want dups.
> >
> > After some more thought, maildrop has a utility (reformail) that makes
> > the above unnecessary. For each user X who wants dups eliminated,
> > ~X/.qmail becomes:
> >
> > | ! reformail -D 50000 .msg-ids || exit 99
> > &X-delivery
> >
> > and ~X/.qmail-delivery contains:
> >
> > ./Mail/inbox/
> >
> > or however you want the non-dup mails handled.
> >
> > Read the reformail and dot-qmail man pages to understand how the above
> > pipeline works.
> >
> >
>
>
--
In a gathering of two or more people, when a lighted cigarette is
placed in an ashtray, the smoke will waft into the face of the non-smoker.
On Sat, Sep 08, 2001 at 05:56:11PM -0400, Jeremy Hansen wrote:
> message a with the same body and subject is coming in at 1 message per
> second, instead of forwarding all this mail that's coming in at this rate,
> send a single message. So I need something that keeps time as well it
> seems.
Sounds like Peter Samuel's duplicate eliminator @ http://www.qmail.org/
fits the bill, then -- just adjust the hardcoded TTL and the headers
checked in the Perl code.
--
Adrian Ho Tinker, Drifter, Fixer, Bum [EMAIL PROTECTED]
ListArchive: <http://marc.theaimsgroup.com/?l=qmail>
Useful URLs: <http://cr.yp.to/qmail.html> <http://www.qmail.org>
<http://www.lifewithqmail.org/> <http://qmail.faqts.com/>
On Sat, 08 Sep 2001, I wrote:
> > False. qmail-send creates a bounce message in the extended format only
> > if the original message was in the extended format.
>
> So it does, and my claim is not "false". qmail cannot reliably bounce
> "extended format" (whatever that is), that's a fact you acknowledged.
> That IS a reliability problem no matter what YOU call it.
Let's revisit the issue. Since it's not a common problem one will
encounter, it deserves a _simple_ fix.
Facts (none of that is new, and they may be incomplete):
- It's not qmail-send's fault someone sent "extended format" in.
- It's qmail-qmqpd's purpose to take arbitrary data in.
- The set of delivery backends that qmail-send can choose from is limited to
the local delivery agent and the qmail-remote SMTP client. Neither is
able to transfer "extended format" under all circumstances:
- qmail-remote can only send ASCII, possibly extended by 0x80...0xff,
and requires a CRLF terminated mail with constraints on the line
format (length in particular).
- qmail-local cannot deliver unterminated mail to mbox format spools.
- qmail-local cannot deliver mail that contains (regexp) "^From " to
mbox format spools.
I assume that "extended format" still requires RFC-822 headers.
Here's my suggestion:
* Strip the mail _body_ from the undeliverable bounce when copying it
into the doublebounce.
Reasons:
1/ Under the assumption (1), the double bounce will be deliverable
across SMTP.
2/ The double bounce defaults to postmaster who has no business with
getting the users' mail bodies.
3/ Since the bounce format is fixed, stripping the mail body can be done
safely.
Consequences:
- Postmaster (or whoever is configured) is immediately notified of the
failed mail + bounce.
- The doublebounce recipient no longer sees private user mail bodies.
- Mail need no longer be discarded because the doublebounce will be
deliverable locally to mbox and across SMTP links.
Comments solicited.
I'm tired of ORBZ scanning/spamming qmail.org (they run qmail
themselves; you'd think they would know better). Some days I get more
spam from ORBZ than all other sources put together. So, I'm going to
block their scans on the SMTP level from now on. If you want to do
this too, and you've configured your system as explained in Dave
Sill's Life with Qmail, here's what you should do:
echo '205.231.149.25:deny' >>/etc/tcp.smtp
qmailctl cdb
--
-russ nelson <[EMAIL PROTECTED]> http://russnelson.com
Crynwr sells support for free software | PGPok | The most basic moral/ethical
521 Pleasant Valley Rd. | +1 315 268 1925 voice | question is who gets to use
Potsdam, NY 13676-3213 | +1 315 268 9201 FAX | force, and when -- ESR
I have added a new domain to my qmail server, but when users try and send
mail from an external client, they receive the error:
(Outlook Express - mail.x.com.au, x being the domain in question)
"The message could not be sent because one of the recipients was rejected by
the server. The rejected e-mail address was '[EMAIL PROTECTED]'. Subject
'Test Message', Account: 'mail', Server: 'mail.x.com.au', Protocol: SMTP,
Server Response: '553 sorry, that domain isn't in my list ofallowed
rcpthosts (#5.7.1)', Port: 25, Secure(SSL): No, Server Error: 553, Error
Number 0x800CCC79"
This error is generated if sending from another domain (eg. via an ISP),
however if I send from within our network (ie within the office) all is OK.
I think that the tcp.smtp file is ok, as other existing domains are ok... it
is only the new/added domain.
The domain is listed in the /var/qmail/contol/rcpthosts &
/var/qmail/control/virtualdomains files, and the server has been rebooted
since the domain was added.
Is there anywhere else that the domain needs to be added? Can anyone point
me in the right direction. (I'm a qmail novice!!)
Thanks..
Charles Widdis.
On Sun, Sep 09, 2001 at 04:26:03PM +1000, Charles Widdis wrote:
> This error is generated if sending from another domain (eg. via an ISP),
> however if I send from within our network (ie within the office) all is OK.
> I think that the tcp.smtp file is ok, as other existing domains are ok... it
> is only the new/added domain.
Post it anyway...
> The domain is listed in the /var/qmail/contol/rcpthosts &
> /var/qmail/control/virtualdomains files, and the server has been rebooted
> since the domain was added.
...and show us the output of qmail-showctl.
--
Adrian Ho Tinker, Drifter, Fixer, Bum [EMAIL PROTECTED]
ListArchive: <http://marc.theaimsgroup.com/?l=qmail>
Useful URLs: <http://cr.yp.to/qmail.html> <http://www.qmail.org>
<http://www.lifewithqmail.org/> <http://qmail.faqts.com/>
