[pfx] Re: Milter multiline header formatting
Wietse Venema via Postfix-users wrote in
<4tqsmy5jfczj...@spike.porcupine.org>:
|Wietse Venema via Postfix-users:
|> Again, Postfix does not store line terminators, not when email comes
|> from UNIX tool with \n, via SMTP with \r\n, or encapsulated as
|> netstrings which uses neither.
|>
|> Instead, Postfix generates line terminators upon output, and until
|> now they are always \n.
|
|In headers that Postfix sends to a milter. I may want to change that
|or at least make it configurable with an appropriate default.
|
|I just checked what happens with bodies. There, the Postfix Milter
|client is hard coded to send lines ending in "\r\n".
I even want to apologise for accusing postfix of having a bug!
I was not looking at libmilter at all, i only have "THE SENDMAIL
MILTER PROTOCOL, VERSION 2", and was blindly assuming all this is
speaking RFC {8,28,53}22.
--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter he cheerfully and one by one
|einen nach dem anderen runter wa.ks himself off
|(By Robert Gernhardt)
___
Postfix-users mailing list -- postfix-users@postfix.org
To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Claus Assmann via Postfix-users wrote in <20240307053606.ga48...@veps.esmtp.org>: |On Wed, Mar 06, 2024, Wietse Venema via Postfix-users wrote: | |>> Again, Postfix does not store line terminators, not when email comes |>> from UNIX tool with \n, via SMTP with \r\n, or encapsulated as |>> netstrings which uses neither. | |> In headers that Postfix sends to a milter. I may want to change that |> or at least make it configurable with an appropriate default. | |AFAICT the libmilter docs define the line endings: | |libmilter/docs/xxfi_header.html | ||headerv |Header field value. The content of the header may | |||include folded white space, i.e., multiple lines| |||with following white space where lines are separated| |||by LF (not CRLF). The trailing line terminator | |||(CRLF) is removed. | |Changing that would probably break compatibility for milters. | |> I just checked what happens with bodies. There, the Postfix Milter |> client is hard coded to send lines ending in "\r\n". | |libmilter/docs/xxfi_body.html | |* End-of-lines are represented as received from SMTP (normally CRLF). Interesting. Read the docs (tm). So postfix is fine out, and the fault is even more on the digesting implementations. One more thank you for the NC note. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Wietse Venema via Postfix-users wrote in <4tqsmy5jfczj...@spike.porcupine.org>: |Wietse Venema via Postfix-users: |> Again, Postfix does not store line terminators, not when email comes |> from UNIX tool with \n, via SMTP with \r\n, or encapsulated as |> netstrings which uses neither. |> |> Instead, Postfix generates line terminators upon output, and until |> now they are always \n. | |In headers that Postfix sends to a milter. I may want to change that |or at least make it configurable with an appropriate default. I think the IETF has to adapt to what is reality and adjust the standard(s maybe even). |I just checked what happens with bodies. There, the Postfix Milter |client is hard coded to send lines ending in "\r\n". That is all i will do today, modifying my MUA to be able to send a message with plain LF's and CR's in it. I am pretty sure that "they" will require that to be treated as WSP, too. Thanks, Wietse Venema. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Claus Assmann via Postfix-users: > On Wed, Mar 06, 2024, Wietse Venema via Postfix-users wrote: > > > > Again, Postfix does not store line terminators, not when email comes > > > from UNIX tool with \n, via SMTP with \r\n, or encapsulated as > > > netstrings which uses neither. > > > In headers that Postfix sends to a milter. I may want to change that > > or at least make it configurable with an appropriate default. > > AFAICT the libmilter docs define the line endings: > > libmilter/docs/xxfi_header.html > > |headerv |Header field value. The content of the header may > | > ||include folded white space, i.e., multiple lines > | > ||with following white space where lines are > separated| > ||by LF (not CRLF). The trailing line terminator > | > ||(CRLF) is removed. > > Changing that would probably break compatibility for milters. > > > I just checked what happens with bodies. There, the Postfix Milter > > client is hard coded to send lines ending in "\r\n". > > libmilter/docs/xxfi_body.html > > * End-of-lines are represented as received from SMTP (normally CRLF). Thanks. I was searching online, and could have found it in the sendmail source. Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
On Wed, Mar 06, 2024, Wietse Venema via Postfix-users wrote: > > Again, Postfix does not store line terminators, not when email comes > > from UNIX tool with \n, via SMTP with \r\n, or encapsulated as > > netstrings which uses neither. > In headers that Postfix sends to a milter. I may want to change that > or at least make it configurable with an appropriate default. AFAICT the libmilter docs define the line endings: libmilter/docs/xxfi_header.html |headerv |Header field value. The content of the header may | ||include folded white space, i.e., multiple lines| ||with following white space where lines are separated| ||by LF (not CRLF). The trailing line terminator | ||(CRLF) is removed. Changing that would probably break compatibility for milters. > I just checked what happens with bodies. There, the Postfix Milter > client is hard coded to send lines ending in "\r\n". libmilter/docs/xxfi_body.html * End-of-lines are represented as received from SMTP (normally CRLF). ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Wietse Venema via Postfix-users: > Again, Postfix does not store line terminators, not when email comes > from UNIX tool with \n, via SMTP with \r\n, or encapsulated as > netstrings which uses neither. > > Instead, Postfix generates line terminators upon output, and until > now they are always \n. In headers that Postfix sends to a milter. I may want to change that or at least make it configurable with an appropriate default. I just checked what happens with bodies. There, the Postfix Milter client is hard coded to send lines ending in "\r\n". Wietse ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Hello Scott Kitterman. Scott Kitterman via Postfix-users wrote in : .. |As far as I know, we're doing it mostly correctly I'm dkimpy (see below). \ | It's used in lots of ways that have nothing to do with postfix, so \ |I am strongly inclined to believe it's right or there would have been \ |lots of reports of interoperability issues. I am up to my eyeballs \ |in other stuff at the moment, so there's no way I can even contemplate \ |time for a code review in the absence of a known issue. | |One thing that may be a factor is the canonicalization algorithm being \ |used. I tried simple early on, but it proved to be fragile. I suspect \ |(but equally haven't recently checked) most are using relaxed canonicali\ |zation. For the header, line endings don't matter for it since you \ |unfold everything. The 'simple' canonicalization is easy to implement, \ |but very fragile. It is, in fact, bad enough that even if you tell \ |dkimpy to use simple for the header, it does relaxed anyway. For the \ |body it will do whichever is specified. I only support relaxed/relaxed (for now). Yes; i do not know, you know? My change was to treat LF and CR outside of CRLF the same as SP and HT, and now it works. Thank you! --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
On March 6, 2024 7:37:47 PM UTC, Steffen Nurpmeso via Postfix-users wrote: >Hello Wietse Venema :) > >Wietse Venema via Postfix-users wrote in > <4tqhxw0ksyzj...@spike.porcupine.org>: > |Steffen Nurpmeso via Postfix-users: > |> Wietse Venema via Postfix-users wrote in > |> <4tqh100n6pzj...@spike.porcupine.org>: > |>|Are you trying to say that Postfix represents a multiline message > |>|header as text with \n instead of \r\n? > |> > |> Yes. > ... > |>|I agree that \r\n would be better, but I'm surprised that this has > |>|not come up before. > .. > |Every Received: header is multiline, and it is not unusual to have > >These are not "usually" DKIM signed i'd say. > > |multiline Subject:, To:, and other common headers. > >Yes. > > |Again, Postfix does not store line terminators, not when email comes > |from UNIX tool with \n, via SMTP with \r\n, or encapsulated as > |netstrings which uses neither. > >OK then there is nothing i can do on the MUA side. > > |Instead, Postfix generates line terminators upon output, and until > |now they are always \n. > | > |Postfix systems have been signing with Milters since Postfix Milter > |support was added in 2006. I'm just surprised that the non-canoncal > |line endings in a multiline header have not been a problem before. > >My milter now treats LF and CR not in a CRLF as real whitespace. >The email i just sent was accepted by Google, this one should also >wrap, and we see what this software does (rspamd is it i think). > >So it seems to me, without having looked what OpenDKIM does, and >maybe i should explicitly Cc: Scott Kitterman for his dkimpy, but >i know he is on this list, just let's wait a bit, that all these >DKIM milters do it falsely, and treat LF and CR other than the >combination of CRLF as whitespace. Even though WSP is defined by >"RFC 5234, B.1. Core Rules:" as "WSP= SP / HT; white space", >and that is what the DKIM and email standards go for. For the >latter i explicitly asked a month or so ago. >So they break the email standard, postfix does, too, and in >combination all is fine again. As far as I know, we're doing it mostly correctly I'm dkimpy (see below). It's used in lots of ways that have nothing to do with postfix, so I am strongly inclined to believe it's right or there would have been lots of reports of interoperability issues. I am up to my eyeballs in other stuff at the moment, so there's no way I can even contemplate time for a code review in the absence of a known issue. One thing that may be a factor is the canonicalization algorithm being used. I tried simple early on, but it proved to be fragile. I suspect (but equally haven't recently checked) most are using relaxed canonicalization. For the header, line endings don't matter for it since you unfold everything. The 'simple' canonicalization is easy to implement, but very fragile. It is, in fact, bad enough that even if you tell dkimpy to use simple for the header, it does relaxed anyway. For the body it will do whichever is specified. Scott K ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Steffen Nurpmeso via Postfix-users wrote in <20240306193747.mAtzRjYs@steffen%sdaoden.eu>: ... |My milter now treats LF and CR not in a CRLF as real whitespace. |The email i just sent was accepted by Google, this one should also |wrap, and we see what this software does (rspamd is it i think). | |So it seems to me, without having looked what OpenDKIM does, and |maybe i should explicitly Cc: Scott Kitterman for his dkimpy, but |i know he is on this list, just let's wait a bit, that all these |DKIM milters do it falsely, and treat LF and CR other than the |combination of CRLF as whitespace. Even though WSP is defined by |"RFC 5234, B.1. Core Rules:" as "WSP= SP / HT; white space", |and that is what the DKIM and email standards go for. For the |latter i explicitly asked a month or so ago. |So they break the email standard, postfix does, too, and in |combination all is fine again. Authentication-Results: list.sys4.de; dkim=pass header.d=sdaoden.eu; arc=none (Message is not ARC signed); dmarc=none Hahahaha! What a mess!!! ;-) Btw it seems Google does not support Ed25519 keys at all?? And if anyone else is writing a DKIM thing. I am thankful i could contact https://www.appmaildev.com/de/dkim for testing purposes. (If you read this: your service is broken, it does not support continuation lines in DKIM signatures *at*all*. But thank you!) --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
[pfx] Re: Milter multiline header formatting
Hello Wietse Venema :) Wietse Venema via Postfix-users wrote in <4tqhxw0ksyzj...@spike.porcupine.org>: |Steffen Nurpmeso via Postfix-users: |> Wietse Venema via Postfix-users wrote in |> <4tqh100n6pzj...@spike.porcupine.org>: |>|Are you trying to say that Postfix represents a multiline message |>|header as text with \n instead of \r\n? |> |> Yes. ... |>|I agree that \r\n would be better, but I'm surprised that this has |>|not come up before. .. |Every Received: header is multiline, and it is not unusual to have These are not "usually" DKIM signed i'd say. |multiline Subject:, To:, and other common headers. Yes. |Again, Postfix does not store line terminators, not when email comes |from UNIX tool with \n, via SMTP with \r\n, or encapsulated as |netstrings which uses neither. OK then there is nothing i can do on the MUA side. |Instead, Postfix generates line terminators upon output, and until |now they are always \n. | |Postfix systems have been signing with Milters since Postfix Milter |support was added in 2006. I'm just surprised that the non-canoncal |line endings in a multiline header have not been a problem before. My milter now treats LF and CR not in a CRLF as real whitespace. The email i just sent was accepted by Google, this one should also wrap, and we see what this software does (rspamd is it i think). So it seems to me, without having looked what OpenDKIM does, and maybe i should explicitly Cc: Scott Kitterman for his dkimpy, but i know he is on this list, just let's wait a bit, that all these DKIM milters do it falsely, and treat LF and CR other than the combination of CRLF as whitespace. Even though WSP is defined by "RFC 5234, B.1. Core Rules:" as "WSP= SP / HT; white space", and that is what the DKIM and email standards go for. For the latter i explicitly asked a month or so ago. So they break the email standard, postfix does, too, and in combination all is fine again. --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt) ___ Postfix-users mailing list -- postfix-users@postfix.org To unsubscribe send an email to postfix-users-le...@postfix.org
