PATCH: Postscreen vs. BDAT
Ralf Hildebrandt:
> > It is also possible that the Exim version in question is out of date,
> > I recall seeing various bug reports on the Exim-users list about the
> > CHUNKING support in Exim, even some security issues. Don't know whether
> > the same symptoms are to be expected from a fully-patched version.
>
> According to the headers smtpgw03.nextwerk.de is running Exim 4.90_1
Also released as part of postfix-3.4-20180903.
Wietse
diff --exclude=man --exclude=html --exclude=README_FILES --exclude=INSTALL
--exclude=.indent.pro --exclude=Makefile.in -r -ur
/var/tmp/postfix-3.4-20180827/src/postscreen/postscreen_smtpd.c
./src/postscreen/postscreen_smtpd.c
--- /var/tmp/postfix-3.4-20180827/src/postscreen/postscreen_smtpd.c
2018-08-25 19:36:13.0 -0400
+++ ./src/postscreen/postscreen_smtpd.c 2018-09-03 18:49:28.0 -0400
@@ -737,6 +737,7 @@
#define PSC_SMTPD_CMD_FLAG_DESTROY (1<<1) /* dangling pointer alert */
#define PSC_SMTPD_CMD_FLAG_PRE_TLS (1<<2) /* allowed with mandatory TLS */
#define PSC_SMTPD_CMD_FLAG_SUSPEND (1<<3) /* suspend command engine */
+#define PSC_SMTPD_CMD_FLAG_HAS_PAYLOAD (1<<4) /* command has payload */
static const PSC_SMTPD_COMMAND command_table[] = {
"HELO", psc_helo_cmd, PSC_SMTPD_CMD_FLAG_ENABLE |
PSC_SMTPD_CMD_FLAG_PRE_TLS,
@@ -749,7 +750,7 @@
"RCPT", psc_rcpt_cmd, PSC_SMTPD_CMD_FLAG_ENABLE,
"DATA", psc_data_cmd, PSC_SMTPD_CMD_FLAG_ENABLE |
PSC_SMTPD_CMD_FLAG_DESTROY,
/* ".", psc_dot_cmd, PSC_SMTPD_CMD_FLAG_NONE, */
-"BDAT", psc_bdat_cmd, PSC_SMTPD_CMD_FLAG_ENABLE |
PSC_SMTPD_CMD_FLAG_DESTROY,
+"BDAT", psc_bdat_cmd, PSC_SMTPD_CMD_FLAG_ENABLE |
PSC_SMTPD_CMD_FLAG_DESTROY | PSC_SMTPD_CMD_FLAG_HAS_PAYLOAD,
"RSET", psc_rset_cmd, PSC_SMTPD_CMD_FLAG_ENABLE,
"NOOP", psc_noop_cmd, PSC_SMTPD_CMD_FLAG_ENABLE |
PSC_SMTPD_CMD_FLAG_PRE_TLS,
"VRFY", psc_vrfy_cmd, PSC_SMTPD_CMD_FLAG_ENABLE,
@@ -1031,7 +1032,8 @@
}
}
/* Command PIPELINING test. */
- if ((state->flags & PSC_STATE_MASK_PIPEL_TODO_SKIP)
+ if ((state->flags & PSC_SMTPD_CMD_FLAG_HAS_PAYLOAD) == 0
+ && (state->flags & PSC_STATE_MASK_PIPEL_TODO_SKIP)
== PSC_STATE_FLAG_PIPEL_TODO && !PSC_SMTPD_BUFFER_EMPTY(state)) {
printable(command, '?');
PSC_SMTPD_ESCAPE_TEXT(psc_temp, PSC_SMTPD_PEEK_DATA(state),
Re: Hostname being set by server, not by main.cf
postfinger gives: On Sun, Sep 2, 2018 at 10:02 PM, Viktor Dukhovni wrote: > > > > On Sep 2, 2018, at 10:30 PM, C. Petro wrote: > > > > What am I missing? > > http://www.postfix.org/DEBUG_README.html > > -- > Viktor. > >
Re: Postscreen vs. BDAT
> It is also possible that the Exim version in question is out of date, > I recall seeing various bug reports on the Exim-users list about the > CHUNKING support in Exim, even some security issues. Don't know whether > the same symptoms are to be expected from a fully-patched version. According to the headers smtpgw03.nextwerk.de is running Exim 4.90_1 -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: Postscreen vs. BDAT
> On Sep 3, 2018, at 10:45 AM, Wietse Venema wrote: > > 2) Exim sends BDAT after RCPT is rejected. That is a protocol error. > There is no pipelining error, because Exim did not pipeline BDAT > after RCPT. postscreen did not log "command pipelining after RCPT". It is perhaps possible that Exim sent BDAT separately from RCPT with a consequent Nagle delay, so that Postfix saw just RCPT TO, but BDAT was sent before Exim saw (or bothered to read the response for) the rejected recipient. It is also possible that the Exim version in question is out of date, I recall seeing various bug reports on the Exim-users list about the CHUNKING support in Exim, even some security issues. Don't know whether the same symptoms are to be expected from a fully-patched version. -- Viktor.
Re: Postscreen vs. BDAT
Viktor Dukhovni: > > > > On Sep 3, 2018, at 9:31 AM, Wietse Venema wrote: > > > > I have difficuty parsing your suggestion that postsccreen should > > not consider this a pipelinig violation. > > > > The client pipelines commands, postscreen does not announce > > pipelining, therefore the client is at fault. > > Perhaps I was too cryptic, my comment was that the pipelining > violation was sending BDAT in the first place, while Ralf's > logs indicate that the pipelining violation was the message > payload after BDAT, which is not in itself a pipeling issue > since BDAT payload succeeds the command without a response > from the server. OK, so the we have two pipelining claims. 1) postscreen's claim that something was pipelined after BDAT. That claim is wrong, because it looks for the FIRST bytes after a BDAT command. I can fix that. (a correct pipelining check would look for bytes pipelined after the LAST bytes of BDAT payload, but that requires processing the entire BDAT payload and is practical only with small payloads). 2) Exim sends BDAT after RCPT is rejected. That is a protocol error. There is no pipelining error, because Exim did not pipeline BDAT after RCPT. postscreen did not log "command pipelining after RCPT". Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=, to=, proto=ESMTP, helo= Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa ( Wietse
Re: Postscreen vs. BDAT
> On Sep 3, 2018, at 9:31 AM, Wietse Venema wrote: > > I have difficuty parsing your suggestion that postsccreen should > not consider this a pipelinig violation. > > The client pipelines commands, postscreen does not announce > pipelining, therefore the client is at fault. Perhaps I was too cryptic, my comment was that the pipelining violation was sending BDAT in the first place, while Ralf's logs indicate that the pipelining violation was the message payload after BDAT, which is not in itself a pipeling issue since BDAT payload succeeds the command without a response from the server. -- -- Viktor.
Re: Postscreen vs. BDAT
Viktor Dukhovni: > /* > * smtpd(8) incompatibility: postscreen(8) drops the connection, instead > * of reading the entire BDAT chunk and staying in sync with the client. > * Justification: postscreen(8) should never see BDAT from a legitimate > * client, because 1) the server rejects every recipient, and 2) the > * server does not announce PIPELINING. > */ > ... > > So I would expect that the BDAT command should not have been sent by the > client. Given that it was sent, Postfix should arguably not report the > subsequent message a pipelining violation, but surely the BDAT command > itself is a violation of the same kind... I have difficuty parsing your suggestion that postsccreen should not consider this a pipelinig violation. The client pipelines commands, postscreen does not announce pipelining, therefore the client is at fault. Wietse
Re: Postscreen vs. BDAT
* Wietse Venema : > Ralf Hildebrandt: > > Today a fellow postmaster (using Exim) called me, they were having problems > > sending > > mail to charite.de. In my log I found: > > > > Sep 3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from > > [31.7.179.105]:38256 to [193.175.73.208]:25 > > Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from > > [31.7.179.105]:38256 > > Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection > > established from [31.7.179.105]:38256: TLSv1.2 with cipher > > ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) > > Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT > > from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; > > from=, to=, proto=ESMTP, > > helo= > > Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from > > [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] > > (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa ( > > > > After I disabled chunking, mail would flow again. > > Is this an Exim issue or a flaw in the newly introduced BDAT feature? > > The client is at fault. Postscreen DOES NOT announce PIPELNING support. Thanks! -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: Postscreen vs. BDAT
Ralf Hildebrandt: > Today a fellow postmaster (using Exim) called me, they were having problems > sending > mail to charite.de. In my log I found: > > Sep 3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from > [31.7.179.105]:38256 to [193.175.73.208]:25 > Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from > [31.7.179.105]:38256 > Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection > established from [31.7.179.105]:38256: TLSv1.2 with cipher > ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) > Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT > from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; > from=, to=, proto=ESMTP, > helo= > Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from > [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] > (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa ( > > After I disabled chunking, mail would flow again. > Is this an Exim issue or a flaw in the newly introduced BDAT feature? The client is at fault. Postscreen DOES NOT announce PIPELNING support. Wietse
Re: Postscreen vs. BDAT
> On Sep 3, 2018, at 8:41 AM, Ralf Hildebrandt wrote:
>
> Today a fellow postmaster (using Exim) called me, they were having problems
> sending
> mail to charite.de. In my log I found:
>
> Sep 3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from
> [31.7.179.105]:38256 to [193.175.73.208]:25
> Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from
> [31.7.179.105]:38256
> Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection
> established from [31.7.179.105]:38256: TLSv1.2 with cipher
> ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
> Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT
> from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable;
> from=, to=, proto=ESMTP,
> helo=
> Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from
> [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75]
> (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa (
>
> After I disabled chunking, mail would flow again.
> Is this an Exim issue or a flaw in the newly introduced BDAT feature?
The "postscreen" service intentionally does not support ESMTP PIPELINING:
posttls-finger "[mail-cbf-ext.charite.de]"
posttls-finger: Connected to mail-cbf-ext.charite.de[193.175.73.208]:25
posttls-finger: < 220-mail-cbf.charite.de ESMTP Postfix
posttls-finger: < 220 mail-cbf.charite.de ESMTP Postfix
posttls-finger: > EHLO amnesiac
posttls-finger: < 250-mail-cbf.charite.de
posttls-finger: < 250-SIZE 36700160
posttls-finger: < 250-ETRN
posttls-finger: < 250-STARTTLS
posttls-finger: < 250-ENHANCEDSTATUSCODES
posttls-finger: < 250 8BITMIME
posttls-finger: > STARTTLS
posttls-finger: < 220 2.0.0 Ready to start TLS
[...]
posttls-finger: Untrusted TLS connection established to
mail-cbf-ext.charite.de[193.175.73.208]:25: TLSv1.2 with cipher
ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
posttls-finger: > EHLO amnesiac
posttls-finger: < 250-mail-cbf.charite.de
posttls-finger: < 250-SIZE 36700160
posttls-finger: < 250-ETRN
posttls-finger: < 250-ENHANCEDSTATUSCODES
posttls-finger: < 250 8BITMIME
posttls-finger: > QUIT
posttls-finger: < 221 2.0.0 Bye
The postscreen source code reads in part:
static int psc_bdat_cmd(PSC_STATE *state, char *args)
{
const char *myname = "psc_bdat_cmd";
/*
* smtpd(8) incompatibility: postscreen(8) drops the connection, instead
* of reading the entire BDAT chunk and staying in sync with the client.
* Justification: postscreen(8) should never see BDAT from a legitimate
* client, because 1) the server rejects every recipient, and 2) the
* server does not announce PIPELINING.
*/
...
So I would expect that the BDAT command should not have been sent by the
client. Given that it was sent, Postfix should arguably not report the
subsequent message a pipelining violation, but surely the BDAT command
itself is a violation of the same kind...
--
Viktor.
Postscreen vs. BDAT
Today a fellow postmaster (using Exim) called me, they were having problems sending mail to charite.de. In my log I found: Sep 3 00:31:18 mail-cbf postfix/postscreen[34943]: CONNECT from [31.7.179.105]:38256 to [193.175.73.208]:25 Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: CONNECT from [31.7.179.105]:38256 Sep 3 00:31:24 mail-cbf postfix/tlsproxy[39995]: Anonymous TLS connection established from [31.7.179.105]:38256: TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits) Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: NOQUEUE: reject: RCPT from [31.7.179.105]:38256: 450 4.3.2 Service currently unavailable; from=, to=, proto=ESMTP, helo= Sep 3 00:31:24 mail-cbf postfix/postscreen[34943]: COMMAND PIPELINING from [31.7.179.105]:38256 after BDAT: Received: from [213.182.136.75] (helo=NWH-EX13.nwcloud.loc)\r\n\tby smtpgw03.nextwerk.de with esmtpsa ( After I disabled chunking, mail would flow again. Is this an Exim issue or a flaw in the newly introduced BDAT feature? -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: [OT] Postfwd question
* Alex JOST : > Sat = 6 > Sun = 0 > > Maybe postwfd has issues dealing with a range of 6-0. Have you tried > specifying both weekdays separately? Nope, I should try that. -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
Re: [OT] Postfwd question
Am 03.09.2018 um 13:15 schrieb Ralf Hildebrandt: id=mass_mailing_feierabend time=17:00:01-23:59:59 time=00:00:00-08:59:59 days=Mon-Fri & action=rcpt(sender/100/43200/450 4.7.1 Recipient limit exceeded) # This is also working, but I feel stupid using these two definitions # for the periods before and after work!! I think you could write it like this: time=!!09:00:00-17:00:00 -- Alex JOST
Re: [OT] Postfwd question
Am 03.09.2018 um 13:15 schrieb Ralf Hildebrandt: I know, I know, it's offtopic since it'S not entirely postfix per se, but I am at my wit's end here. I'm trying to implement a (I think) simple ratelimiting feature: * during our business hours 400 Mails per sender from internat host * otherwise 100 # sonst id=mass_mailing_wochenende time=00:00:00-23:59:59 days=Sat-Sun & action=rcpt(sender/100/43200/450 4.7.1 Recipient limit exceeded) # Alas, this is not triggering at all. Dunno why! Sat = 6 Sun = 0 Maybe postwfd has issues dealing with a range of 6-0. Have you tried specifying both weekdays separately? -- Alex JOST
[OT] Postfwd question
I know, I know, it's offtopic since it'S not entirely postfix per se, but I am at my wit's end here. I'm trying to implement a (I think) simple ratelimiting feature: * during our business hours 400 Mails per sender from internat host * otherwise 100 Some of my limits work, others don't trigger at all: id=mass_mailing_exceptions & sender==file:/etc/postfix/mass_mailing_absolute_exceptions action=dunno # these are exceptions for high volume senders. Working OK! # Arbeitszeit id=mass_mailing_business_hours days=Mon-Fri time=09:00:00-17:00:00 & action=rcpt(sender/400/43200/450 4.7.1 Recipient limit exceeded) # Monday to friday, 9 to 5, working as it should id=mass_mailing_feierabend time=17:00:01-23:59:59 time=00:00:00-08:59:59 days=Mon-Fri & action=rcpt(sender/100/43200/450 4.7.1 Recipient limit exceeded) # This is also working, but I feel stupid using these two definitions # for the periods before and after work!! # sonst id=mass_mailing_wochenende time=00:00:00-23:59:59 days=Sat-Sun & action=rcpt(sender/100/43200/450 4.7.1 Recipient limit exceeded) # Alas, this is not triggering at all. Dunno why! -- [*] sys4 AG https://sys4.de, +49 (89) 30 90 46 64 Schleißheimer Straße 26/MG, 80333 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein
