If qmail-smtpd gets a timeout it simply issues the 451 and exits without
invoking qmail-queue. To pretend it had got a <CRLF>.<CRLF> would break RFC 821
and 2821. Here's exactly what it does:
--------- qmail-sptpd.c snip --------------
void die_alarm() { out("451 timeout (#4.4.2)\r\n"); flush(); _exit(1); }
int saferead(fd,buf,len) int fd; char *buf; int len;
{
............
r = timeoutread(timeout,fd,buf,len);
if (r == -1) if (errno == error_timeout) die_alarm();
............
}
---------------------------------------------------
This is all done from within the DATA function "blast()" (I'm not sure if
that's explosive or invective :-))
Incidentally RFC 2821 states:
"The custom of accepting lines ending only in <LF>, as a concession to
non-conforming behavior on the part of some UNIX systems, has proven
to cause more interoperability problems than it solves, and SMTP
server systems MUST NOT do this..."
I'm surprised you can't test qmail with telnet. I used to do it quite regularly
to check out users' complaints. Now we use SMTPAUTH it's too much of a pain to
type all that encoded username/password stuff.
Chris Robinson
<CRLF>.<CRLF>
----- Original Message -----
From: Sam Clippinger
To: spamdyke users
Sent: Friday, February 08, 2008 07:02
Subject: Re: [spamdyke-users] Spamdyke passes partial emails to qmailafter
timeout
Well as I said, I assumed qmail would send the first part of the message
if the remote server disconnected without completing it but I never
tested it. If that's not what qmail does, I'll remove it from spamdyke.
Inserting carriage returns is a different kettle of fish, however. I
personally disagree with DJB's position about strictly interpreting the
RFCs -- I believe software should strictly follow RFCs when producing
output and loosely follow them when accepting it. I think his position
on this has created more problems than it has solved. Among other
things, it really annoys me that I can't test a qmail server with a
telnet session.
There are only two situations I can imagine where inserting the carriage
returns could possibly cause a problem. First, ESMTP allows the sender
to specify the size of the message body within the DATA command.
Inserting carriage returns may cause the message to be bigger than the
number indicates, which may cause the mail server to truncate the
message. Second, cryptographic software (e.g. PGP, GPG) work by
producing a checksum from the message content. Inserting carriage
returns may invalidate the signature. I haven't tested either of those
things.
-- Sam Clippinger
Chris Robinson wrote:
> OK that explains everything.
>
> I honestly admire spamdyke enormously; for its clean, readable and
> efficient code, and for the way it turned my servers into racehorses
> rather than donkeys labouring under a load average around 8 to 10. I
> don't know what I'd do without it now.
>
> But I must honestly say that I think spamdyke should not interfere with
> the data stream once it's done it's job on the envelope and headers. The
> first thing in the README is the About Spamdyke which says ... "It acts
> as a transparent middleman, observing the conversation without
> interference" ..., and I feel it should honour this strictly. To be
> honest I'm even have my doubts about it "silently inserting a carriage
> return before any bare line feed characters it sees".
>
> Maybe all this could be an option such as "strict-transparency" in the
> config file?
>
> Chris Robinson
>
> ----- Original Message -----
> *From:* Sam Clippinger <mailto:[EMAIL PROTECTED]>
> *To:* spamdyke users <mailto:spamdyke-users@spamdyke.org>
> *Sent:* Thursday, February 07, 2008 17:14
> *Subject:* Re: [spamdyke-users] Spamdyke passes partial emails to
> qmailafter timeout
>
> The changes to sendrecv aren't related to this -- sendrecv is a program
> I wrote only for testing spamdyke (as part of the testing scripts). It
> isn't involved in spamdyke's operation at all. The changes to sendrecv
> in 3.1.4 were needed because spamdyke was corrupting the SSL handshake.
> sendrecv contained a matching bug that covered the corruption and
> allowed the SSL handshake to proceed correctly.
>
> You're seeing this behavior because when spamdyke disconnects a remote
> client due to a timeout (either idle timeout or maximum connection
> time), it sends a terminating dot and a "QUIT" command to qmail. This
> was a judgement call on my part -- I felt that it was better to close
> qmail correctly and deliver whatever data had been sent than to
> potentially lose it if the client didn't retry. Not all clients
> correctly handle errors during/after the DATA command.
>
> I also assumed qmail already behaved this way, though I didn't test it.
> If that assumption is wrong and this is causing problems, I can
> easily
> change it.
>
> -- Sam Clippinger
>
> Chris Robinson wrote:
> > Thanks Hartmut,
> >
> > I think you may be right about the sendrecv changes, specially if
> the
> > child process somehow closed the connection to qmail in a way that
> > satisfied qmail-smtpd. Could it be that the child was actually
> sending
> > the SMTP terminating dot without receiving it from the departed
> client?
> >
> > Whatever.
> >
> > Chris
> >
> > ----- Original Message -----
> > *From:* Andreas <mailto:[EMAIL PROTECTED]>
> > *To:* spamdyke users <mailto:spamdyke-users@spamdyke.org>
> > *Sent:* Thursday, February 07, 2008 14:15
> > *Subject:* Re: [spamdyke-users] Spamdyke passes partial emails to
> > qmailafter timeout
> >
> > Hi Hartmut,
> >
> > I had the same phenomen on 1 of my qmail-servers as long as I
> had a
> > nameserver related problem.
> > I had up to 750 processes of spamdyke.
> > I had to reboot the server to get rid of those processes,
> just killing
> > them was not enough, they immediately reappeared.
> > Sind my nemserver works properly I have, on the same machine,
> just 10-20
> > processes of spamdyke and it works fine now.
> > My spamdyke- version is 3.15.
> > Bye
> >
> > Andreas
> > Am Donnerstag, den 07.02.2008, 08:36 +0100 schrieb Hartmut
> Wernisch:
> > > I hade customers which got emails again and again in
> spamdyke version
> > > prior to 3.1.4. The maximum timeout I have been setting
> was 600.
> > > Whitelisting didn't help.
> > >
> > > Version 3.1.4 of spamdyke fixed this behaviour for me. Maybe
> > because of
> > > the changes of sendrecv in release 3.1.4?
> > >
> > > On 06 Feb 08, Chris Robinson wrote:
> > > > Maybe I didn't express myself properly. Spamdyke was timeing
> > out the
> > > > connection perfectly correctly because the client was
> inactive
> > for more than
> > > > 60 seconds which was spamdyke's default idle timeout. So the
> > client's
> > > > Outlook sees the disconnection and retries, often
> resulting in
> > another
> > > > partial copy of the same message. That is why I've probably
> > fixed it by
> > > > setting "idle-timeout-secs=1200", which should handle
> even the
> > slowest
> > > > clients.
> > > >
> > > > But it's not the cause, it's the effect that concerns
> me. The
> > partial
> > > > message received before the disconnection (e.g. 50 Kb of a
> > 200Kb email) has
> > > > been piped to qmail-smtpd which then actually delivers
> it. How
> > could
> > > > qmail-smtpd deliver a message that has not been properly
> > terminated? Does
> > > > spamdyke simply close the pipe to qmail-smtpd, as would
> happen if
> > > > qmail-smtpd were connected directly to the client and
> > qmail-smtpd initiated
> > > > the timeout?
> > > >
> > > > Somehow a fundamental principle of SMTP MTAs is being
> breached,
> > namely that
> > > > an incomplete session will never result in delivery,
> local or
> > remote, of a
> > > > partial message.
> > > >
> > > > My details:
> > > >
> > > > xinetd.d
> > > > ---------
> > > > server = /var/qmail/bin/tcp-env
> > > > server_args = -Rt0 /usr/local/bin/spamdyke -f
> > > > /usr/np/mail/spamdyke/spamdyke.conf
> /var/qmail/bin/qmail-smtpdauth
> > > > /var/qmail/bin/checkpassword /bin/true
> > > >
> > > > Clamav and Spamassassin are only run after qmail-local
> pipes it
> > to procmail
> > > > via the .qmail file, so they can't be affecting the
> issue. Of
> > course neither
> > > > of those programs are run when the recipient is remote and
> > qmail-remote
> > > > handles it. But even remote recipients are receiving
> multiple,
> > partial
> > > > copies of the same message
> > > >
> > > > Cheers and thanks for the response,
> > > > Chris Robinson
> > > >
> > > > ----- Original Message -----
> > > > From: "Sam Clippinger" <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED]>>
> > > > To: "spamdyke users" <spamdyke-users@spamdyke.org
> <mailto:spamdyke-users@spamdyke.org>
> > <mailto:spamdyke-users@spamdyke.org>>
> > > > Sent: Wednesday, February 06, 2008 18:11
> > > > Subject: Re: [spamdyke-users] Spamdyke passes partial
> emails to
> > qmail after
> > > > timeout
> > > >
> > > >
> > > > > Most likely, this is a virus/spam filter issue. Some
> qmail
> > > > > installations pass the incoming email to Spamassassin or
> > ClamAV before
> > > > > acknowledging the delivery. When that happens, spamdyke's
> > idle timeout
> > > > > can trigger a disconnection if the scanner takes too long
> > (this is
> > > > > common for large attachments).
> > > > >
> > > > > The best way to be sure is to enable full logging (with
> > "full-log-dir")
> > > > > and examine the logs for disconnected deliveries. The log
> > will contain
> > > > > timestamps that will show long each response was delayed.
> > > > >
> > > > > Of course, it's also possible you've found a bug. :)
> In that
> > case, I'll
> > > > > need to find a way to reproduce it, so I may need you to
> > (privately)
> > > > > send me a few full logs along with details of your
> spamdyke
> > configuration.
> > > > >
> > > > > -- Sam Clippinger
> > > > >
> > > > > Chris Robinson wrote:
> > > > > > Hi Everyone,
> > > > > >
> > > > > > I run mail servers for about 30 companies (qmail
> /spamdyke
> > 3.1.0 /
> > > > Fedora
> > > > > > 2.4.21) and started experiencing a problem whereby some
> > users complained
> > > > > > that they could send an email and it was received by the
> > recipient
> > > > corrupted
> > > > > > multiple times in varying sizes until eventually the
> final
> > version would
> > > > > > arrive correct in full. To cut a long story short I
> > eventually tracked
> > > > it
> > > > > > down from a string in a user's Outlook log "Talk faster
> > next time". A
> > > > google
> > > > > > revealed all. It wasn't coming from qmail but from
> > spamdyke, which
> > > > explained
> > > > > > why I couldn't grep it in the qmail source.
> > > > > >
> > > > > > I've probably fixed it by setting
> "idle-timeout-secs=1200".
> > But what
> > > > worries
> > > > > > me is why the recipient got anything except the
> final good
> > email. If
> > > > > > spamdyke issues that 421 error then breaks the
> connection,
> > the user's
> > > > > > Outlook can justifiably assume that the email won't have
> > been accepted
> > > > by
> > > > > > the server, nor sent to the recipient. But what
> seems to
> > happen is that
> > > > the
> > > > > > part that has been received down the pipe so far has
> been
> > passed by
> > > > spamdyke
> > > > > > to qmail-smtpd which has passed it to qmail-queue and
> > thence to the
> > > > local or
> > > > > > remote recipient. Surely no "250 OK 123 qp 456" has come
> > out of qmail?
> > > > > >
> > > > > > Is this a spamdyke issue or a qmail issue?
> > > > > >
> > > > > > Cheers,
> > > > > > Chris Robinson
> > > > > >
> > > > > >
> > > > > >
> > > > > > _______________________________________________
> > > > > > spamdyke-users mailing list
> > > > > > spamdyke-users@spamdyke.org
> <mailto:spamdyke-users@spamdyke.org>
> > <mailto:spamdyke-users@spamdyke.org>
> > > > > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users
> > > > > _______________________________________________
> > > > > spamdyke-users mailing list
> > > > > spamdyke-users@spamdyke.org
> <mailto:spamdyke-users@spamdyke.org>
> <mailto:spamdyke-users@spamdyke.org>
> > > > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users
> > > > >
> > > >
> > > > _______________________________________________
> > > > spamdyke-users mailing list
> > > > spamdyke-users@spamdyke.org
> <mailto:spamdyke-users@spamdyke.org>
> <mailto:spamdyke-users@spamdyke.org>
> > > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users
> > > >
> > > _______________________________________________
> > > spamdyke-users mailing list
> > > spamdyke-users@spamdyke.org
> <mailto:spamdyke-users@spamdyke.org>
> <mailto:spamdyke-users@spamdyke.org>
> > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users
> > >
> >
> > _______________________________________________
> > spamdyke-users mailing list
> > spamdyke-users@spamdyke.org
> <mailto:spamdyke-users@spamdyke.org>
> <mailto:spamdyke-users@spamdyke.org>
> > http://www.spamdyke.org/mailman/listinfo/spamdyke-users
> >
> >
> >
> ------------------------------------------------------------------------
> >
> > _______________________________________________
> > spamdyke-users mailing list
> > spamdyke-users@spamdyke.org <mailto:spamdyke-users@spamdyke.org>
> > http://www.spamdyke.org/mailman/listinfo/spamdyke-users
> _______________________________________________
> spamdyke-users mailing list
> spamdyke-users@spamdyke.org <mailto:spamdyke-users@spamdyke.org>
> http://www.spamdyke.org/mailman/listinfo/spamdyke-users
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> spamdyke-users mailing list
> spamdyke-users@spamdyke.org
> http://www.spamdyke.org/mailman/listinfo/spamdyke-users
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
_______________________________________________
spamdyke-users mailing list
spamdyke-users@spamdyke.org
http://www.spamdyke.org/mailman/listinfo/spamdyke-users
