I keep saying I can't test qmail with telnet -- I mean to say the command line. I can't run qmail-smtpd from the command line and give it a message. Piping a text file through it can also fail if the file isn't in "DOS" format. That's one of the first reasons I wrote the "sendrecv" program that's used by the spamdyke testing scripts.
Sorry for the misstatement. -- Sam Clippinger Sam Clippinger wrote: > Since I've received several complaints about the "." insertion when > timeouts occur, I'll remove it. It was an error on my part. > > As for inserting carriage returns, I stand by what I said before. Since > RFC 2821 doesn't elaborate on the additional problems that have been > created, I can't take its vague statement at face value. RFCs are not > the law, they are documents written to describe how the internet should > work in a perfect world. As a result, many SMTP servers violate RFCs in > order to accommodate badly written clients. It's a matter of picking > your battles; trying to force everyone to send carriage returns by > rejecting their email doesn't work. It only punishes innocent users who > don't know or care about RFCs. I don't believe inserting carriage > returns causes any problems. If anyone can demonstrate that it does, > I'll remove the feature immediately. > > However, if you feel strongly about it, you are free to remove it. Just > change line 339 in spamdyke.c to an "else" instead of an "else if". > Then remove the final "else" block starting on line 344. That's it. > > -- Sam Clippinger > > PS: I can't test qmail from a telnet prompt because Linux doesn't end > its lines with CRLF, just LF. I can issue the HELO, MAIL, RCPT and DATA > commands but as soon as I try to type any message text, qmail cuts me > off with an error message. > > Chris Robinson wrote: >> If qmail-smtpd gets a timeout it simply issues the 451 and exits without >> invoking qmail-queue. To pretend it had got a <CRLF>.<CRLF> would break >> RFC 821 and 2821. Here's exactly what it does: >> >> --------- qmail-sptpd.c snip -------------- >> >> void die_alarm() { out("451 timeout (#4.4.2)\r\n"); flush(); _exit(1); } >> >> int saferead(fd,buf,len) int fd; char *buf; int len; >> { >> ............ >> r = timeoutread(timeout,fd,buf,len); >> if (r == -1) if (errno == error_timeout) die_alarm(); >> ............ >> } >> >> --------------------------------------------------- >> >> This is all done from within the DATA function "blast()" (I'm not sure >> if that's explosive or invective :-)) >> >> Incidentally RFC 2821 states: >> >> "The custom of accepting lines ending only in <LF>, as a concession to >> non-conforming behavior on the part of some UNIX systems, has proven >> to cause more interoperability problems than it solves, and SMTP >> server systems MUST NOT do this..." >> >> I'm surprised you can't test qmail with telnet. I used to do it quite >> regularly to check out users' complaints. Now we use SMTPAUTH it's too >> much of a pain to type all that encoded username/password stuff. >> >> Chris Robinson >> <CRLF>.<CRLF> >> >> ----- Original Message ----- >> *From:* Sam Clippinger <mailto:[EMAIL PROTECTED]> >> *To:* spamdyke users <mailto:spamdyke-users@spamdyke.org> >> *Sent:* Friday, February 08, 2008 07:02 >> *Subject:* Re: [spamdyke-users] Spamdyke passes partial emails to >> qmailafter timeout >> >> Well as I said, I assumed qmail would send the first part of the >> message >> if the remote server disconnected without completing it but I never >> tested it. If that's not what qmail does, I'll remove it from spamdyke. >> >> Inserting carriage returns is a different kettle of fish, however. I >> personally disagree with DJB's position about strictly interpreting the >> RFCs -- I believe software should strictly follow RFCs when producing >> output and loosely follow them when accepting it. I think his position >> on this has created more problems than it has solved. Among other >> things, it really annoys me that I can't test a qmail server with a >> telnet session. >> >> There are only two situations I can imagine where inserting the >> carriage >> returns could possibly cause a problem. First, ESMTP allows the sender >> to specify the size of the message body within the DATA command. >> Inserting carriage returns may cause the message to be bigger than the >> number indicates, which may cause the mail server to truncate the >> message. Second, cryptographic software (e.g. PGP, GPG) work by >> producing a checksum from the message content. Inserting carriage >> returns may invalidate the signature. I haven't tested either of those >> things. >> >> -- Sam Clippinger >> >> Chris Robinson wrote: >> > OK that explains everything. >> > >> > I honestly admire spamdyke enormously; for its clean, readable and >> > efficient code, and for the way it turned my servers into racehorses >> > rather than donkeys labouring under a load average around 8 to 10. I >> > don't know what I'd do without it now. >> > >> > But I must honestly say that I think spamdyke should not >> interfere with >> > the data stream once it's done it's job on the envelope and >> headers. The >> > first thing in the README is the About Spamdyke which says ... >> "It acts >> > as a transparent middleman, observing the conversation without >> > interference" ..., and I feel it should honour this strictly. To be >> > honest I'm even have my doubts about it "silently inserting a >> carriage >> > return before any bare line feed characters it sees". >> > >> > Maybe all this could be an option such as "strict-transparency" >> in the >> > config file? >> > >> > Chris Robinson >> > >> > ----- Original Message ----- >> > *From:* Sam Clippinger <mailto:[EMAIL PROTECTED]> >> > *To:* spamdyke users <mailto:spamdyke-users@spamdyke.org> >> > *Sent:* Thursday, February 07, 2008 17:14 >> > *Subject:* Re: [spamdyke-users] Spamdyke passes partial emails to >> > qmailafter timeout >> > >> > The changes to sendrecv aren't related to this -- sendrecv is >> a program >> > I wrote only for testing spamdyke (as part of the testing >> scripts). It >> > isn't involved in spamdyke's operation at all. The changes >> to sendrecv >> > in 3.1.4 were needed because spamdyke was corrupting the SSL >> handshake. >> > sendrecv contained a matching bug that covered the >> corruption and >> > allowed the SSL handshake to proceed correctly. >> > >> > You're seeing this behavior because when spamdyke disconnects >> a remote >> > client due to a timeout (either idle timeout or maximum >> connection >> > time), it sends a terminating dot and a "QUIT" command to >> qmail. This >> > was a judgement call on my part -- I felt that it was better >> to close >> > qmail correctly and deliver whatever data had been sent than to >> > potentially lose it if the client didn't retry. Not all clients >> > correctly handle errors during/after the DATA command. >> > >> > I also assumed qmail already behaved this way, though I >> didn't test it. >> > If that assumption is wrong and this is causing problems, I can >> > easily >> > change it. >> > >> > -- Sam Clippinger >> > >> > Chris Robinson wrote: >> > > Thanks Hartmut, >> > > >> > > I think you may be right about the sendrecv changes, >> specially if >> > the >> > > child process somehow closed the connection to qmail in a >> way that >> > > satisfied qmail-smtpd. Could it be that the child was actually >> > sending >> > > the SMTP terminating dot without receiving it from the >> departed >> > client? >> > > >> > > Whatever. >> > > >> > > Chris >> > > >> > > ----- Original Message ----- >> > > *From:* Andreas <mailto:[EMAIL PROTECTED]> >> > > *To:* spamdyke users <mailto:spamdyke-users@spamdyke.org> >> > > *Sent:* Thursday, February 07, 2008 14:15 >> > > *Subject:* Re: [spamdyke-users] Spamdyke passes >> partial emails to >> > > qmailafter timeout >> > > >> > > Hi Hartmut, >> > > >> > > I had the same phenomen on 1 of my qmail-servers as >> long as I >> > had a >> > > nameserver related problem. >> > > I had up to 750 processes of spamdyke. >> > > I had to reboot the server to get rid of those processes, >> > just killing >> > > them was not enough, they immediately reappeared. >> > > Sind my nemserver works properly I have, on the same >> machine, >> > just 10-20 >> > > processes of spamdyke and it works fine now. >> > > My spamdyke- version is 3.15. >> > > Bye >> > > >> > > Andreas >> > > Am Donnerstag, den 07.02.2008, 08:36 +0100 schrieb Hartmut >> > Wernisch: >> > > > I hade customers which got emails again and again in >> > spamdyke version >> > > > prior to 3.1.4. The maximum timeout I have been setting >> > was 600. >> > > > Whitelisting didn't help. >> > > > >> > > > Version 3.1.4 of spamdyke fixed this behaviour for >> me. Maybe >> > > because of >> > > > the changes of sendrecv in release 3.1.4? >> > > > >> > > > On 06 Feb 08, Chris Robinson wrote: >> > > > > Maybe I didn't express myself properly. Spamdyke >> was timeing >> > > out the >> > > > > connection perfectly correctly because the client was >> > inactive >> > > for more than >> > > > > 60 seconds which was spamdyke's default idle >> timeout. So the >> > > client's >> > > > > Outlook sees the disconnection and retries, often >> > resulting in >> > > another >> > > > > partial copy of the same message. That is why >> I've probably >> > > fixed it by >> > > > > setting "idle-timeout-secs=1200", which should handle >> > even the >> > > slowest >> > > > > clients. >> > > > > >> > > > > But it's not the cause, it's the effect that concerns >> > me. The >> > > partial >> > > > > message received before the disconnection (e.g. >> 50 Kb of a >> > > 200Kb email) has >> > > > > been piped to qmail-smtpd which then actually >> delivers >> > it. How >> > > could >> > > > > qmail-smtpd deliver a message that has not been >> properly >> > > terminated? Does >> > > > > spamdyke simply close the pipe to qmail-smtpd, as >> would >> > happen if >> > > > > qmail-smtpd were connected directly to the client and >> > > qmail-smtpd initiated >> > > > > the timeout? >> > > > > >> > > > > Somehow a fundamental principle of SMTP MTAs is being >> > breached, >> > > namely that >> > > > > an incomplete session will never result in delivery, >> > local or >> > > remote, of a >> > > > > partial message. >> > > > > >> > > > > My details: >> > > > > >> > > > > xinetd.d >> > > > > --------- >> > > > > server = /var/qmail/bin/tcp-env >> > > > > server_args = -Rt0 /usr/local/bin/spamdyke -f >> > > > > /usr/np/mail/spamdyke/spamdyke.conf >> > /var/qmail/bin/qmail-smtpdauth >> > > > > /var/qmail/bin/checkpassword /bin/true >> > > > > >> > > > > Clamav and Spamassassin are only run after >> qmail-local >> > pipes it >> > > to procmail >> > > > > via the .qmail file, so they can't be affecting the >> > issue. Of >> > > course neither >> > > > > of those programs are run when the recipient is >> remote and >> > > qmail-remote >> > > > > handles it. But even remote recipients are receiving >> > multiple, >> > > partial >> > > > > copies of the same message >> > > > > >> > > > > Cheers and thanks for the response, >> > > > > Chris Robinson >> > > > > >> > > > > ----- Original Message ----- >> > > > > From: "Sam Clippinger" <[EMAIL PROTECTED] >> <mailto:[EMAIL PROTECTED]> >> > <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED]>> >> > > > > To: "spamdyke users" <spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > > <mailto:spamdyke-users@spamdyke.org>> >> > > > > Sent: Wednesday, February 06, 2008 18:11 >> > > > > Subject: Re: [spamdyke-users] Spamdyke passes partial >> > emails to >> > > qmail after >> > > > > timeout >> > > > > >> > > > > >> > > > > > Most likely, this is a virus/spam filter >> issue. Some >> > qmail >> > > > > > installations pass the incoming email to >> Spamassassin or >> > > ClamAV before >> > > > > > acknowledging the delivery. When that happens, >> spamdyke's >> > > idle timeout >> > > > > > can trigger a disconnection if the scanner >> takes too long >> > > (this is >> > > > > > common for large attachments). >> > > > > > >> > > > > > The best way to be sure is to enable full >> logging (with >> > > "full-log-dir") >> > > > > > and examine the logs for disconnected >> deliveries. The log >> > > will contain >> > > > > > timestamps that will show long each response >> was delayed. >> > > > > > >> > > > > > Of course, it's also possible you've found a >> bug. :) >> > In that >> > > case, I'll >> > > > > > need to find a way to reproduce it, so I may >> need you to >> > > (privately) >> > > > > > send me a few full logs along with details of your >> > spamdyke >> > > configuration. >> > > > > > >> > > > > > -- Sam Clippinger >> > > > > > >> > > > > > Chris Robinson wrote: >> > > > > > > Hi Everyone, >> > > > > > > >> > > > > > > I run mail servers for about 30 companies (qmail >> > /spamdyke >> > > 3.1.0 / >> > > > > Fedora >> > > > > > > 2.4.21) and started experiencing a problem >> whereby some >> > > users complained >> > > > > > > that they could send an email and it was >> received by the >> > > recipient >> > > > > corrupted >> > > > > > > multiple times in varying sizes until >> eventually the >> > final >> > > version would >> > > > > > > arrive correct in full. To cut a long story >> short I >> > > eventually tracked >> > > > > it >> > > > > > > down from a string in a user's Outlook log >> "Talk faster >> > > next time". A >> > > > > google >> > > > > > > revealed all. It wasn't coming from qmail but >> from >> > > spamdyke, which >> > > > > explained >> > > > > > > why I couldn't grep it in the qmail source. >> > > > > > > >> > > > > > > I've probably fixed it by setting >> > "idle-timeout-secs=1200". >> > > But what >> > > > > worries >> > > > > > > me is why the recipient got anything except the >> > final good >> > > email. If >> > > > > > > spamdyke issues that 421 error then breaks the >> > connection, >> > > the user's >> > > > > > > Outlook can justifiably assume that the email >> won't have >> > > been accepted >> > > > > by >> > > > > > > the server, nor sent to the recipient. But what >> > seems to >> > > happen is that >> > > > > the >> > > > > > > part that has been received down the pipe so >> far has >> > been >> > > passed by >> > > > > spamdyke >> > > > > > > to qmail-smtpd which has passed it to >> qmail-queue and >> > > thence to the >> > > > > local or >> > > > > > > remote recipient. Surely no "250 OK 123 qp >> 456" has come >> > > out of qmail? >> > > > > > > >> > > > > > > Is this a spamdyke issue or a qmail issue? >> > > > > > > >> > > > > > > Cheers, >> > > > > > > Chris Robinson >> > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > _______________________________________________ >> > > > > > > spamdyke-users mailing list >> > > > > > > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > > <mailto:spamdyke-users@spamdyke.org> >> > > > > > > >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > > > > > _______________________________________________ >> > > > > > spamdyke-users mailing list >> > > > > > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > > > > > >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > > > > > >> > > > > >> > > > > _______________________________________________ >> > > > > spamdyke-users mailing list >> > > > > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > > > > >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > > > > >> > > > _______________________________________________ >> > > > spamdyke-users mailing list >> > > > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > > > >> > > >> > > _______________________________________________ >> > > spamdyke-users mailing list >> > > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > <mailto:spamdyke-users@spamdyke.org> >> > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > > >> > > >> > > >> > >> ------------------------------------------------------------------------ >> > > >> > > _______________________________________________ >> > > spamdyke-users mailing list >> > > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> <mailto:spamdyke-users@spamdyke.org> >> > > http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > _______________________________________________ >> > spamdyke-users mailing list >> > spamdyke-users@spamdyke.org >> <mailto:spamdyke-users@spamdyke.org> >> <mailto:spamdyke-users@spamdyke.org> >> > http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> > >> > >> > >> ------------------------------------------------------------------------ >> > >> > _______________________________________________ >> > spamdyke-users mailing list >> > spamdyke-users@spamdyke.org <mailto:spamdyke-users@spamdyke.org> >> > http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> _______________________________________________ >> spamdyke-users mailing list >> spamdyke-users@spamdyke.org <mailto:spamdyke-users@spamdyke.org> >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> spamdyke-users mailing list >> spamdyke-users@spamdyke.org >> http://www.spamdyke.org/mailman/listinfo/spamdyke-users > _______________________________________________ > spamdyke-users mailing list > spamdyke-users@spamdyke.org > http://www.spamdyke.org/mailman/listinfo/spamdyke-users _______________________________________________ spamdyke-users mailing list spamdyke-users@spamdyke.org http://www.spamdyke.org/mailman/listinfo/spamdyke-users