Re: (Not-so) hypothetical question: What to do about NULs?

2023-03-12 Thread David Levine
I wrote: > Ken wrote: > > In terms of the networking code, > > it looks like the right thing will happen when sending a NUL via > > SMTP, > > Almost, but not quite. I posted a possible fix but I'm still refining > it. Fix to post(8) committed: commit 8f897f65fecbc668db777e2f4fabb23a08edf11b

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-23 Thread Michael Richardson
Ken Hornstein wrote: > Right, but that's mostly because of the way multiline responses are > handled in POP. It's never "read X bytes", it's "read lines until you > get a line that is just .\r\n". With IMAP, it's "the next X bytes are > the data you asked for". So you're used

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-22 Thread Andy Bradford
Thus said Ken Hornstein on Wed, 22 Feb 2023 20:59:31 -0500: > I had an inkling popular MTAs would DTRT. Well, qmail's hardly "popular" these days, but Professor Bernstein had a penchant to make string handling robust to avoid exploits, so he got NUL handling as a benefit. I run minority MTA

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-22 Thread Ken Hornstein
>While POP's LIST does actually include the size of the message in bytes, >that's prior to any CRLF mangling that happens so it cannot be used >reliably as a method for determining when to stop reading. Unfortunate. Right, but that's mostly because of the way multiline responses are handled

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-22 Thread Andy Bradford
Thus said Ken Hornstein on Tue, 21 Feb 2023 21:29:16 -0500: > So you're told "I am sending this many bytes exactly", and you don't > have to deal with "lines", so the implementations I've seen tend to > call read() (or the equivalent) until they get the correct number of > bytes, and

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-22 Thread David Malone
> > I wonder if it would be better to use fwrite instead of write, to > > avoid mixing stdio and Posix-style output? (It would also avoid an > > unbuffered write of 1 byte.) > Good point. How about the attached? Looks sensible to me! David.

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread David Levine
Ken wrote: > [David:] > >I have received email with C-T-E set to binary. While I don't think it > >was needed, I haven't checked closely. > > Facinating! I am curious: who/what sent this to you! Do you remember > the MIME type? The C-T-E: binary is in the message header. The are two

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Ken Hornstein
>When I was poking around in the POP code I didn't notice any special >handling of NUL bytes. It's possible that this would result in >truncation. If that's what we do now, I suspect it's alright to continue >to do so; at least until we find legitimate emails in the wild that do

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Andy Bradford
Thus said Ken Hornstein on Tue, 21 Feb 2023 07:17:19 -0500: > I'm sitting down to write or modify nmh code. Right now we have a lot > of code that assumes NUL-terminated C strings are safe to represent > email everywhere. My question is: is that a valid assumption? I don't think nmh should

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Andy Bradford
Thus said Ken Hornstein on Mon, 20 Feb 2023 21:11:48 -0500: > Facinating! I am curious: who/what sent this to you! Do you remember > the MIME type? 0.11 % (percent) of my messages have Content-Transfer-Encoding of binary at the beginning of the line somewhere in the message. Here are the

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Michael Richardson
Ken Hornstein wrote: > I'm sitting down to write or modify nmh code. Right now we have a lot > of code that assumes NUL-terminated C strings are safe to represent > email everywhere. My question is: is that a valid assumption? If > we are making that assumption, fine, let's be

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Ken Hornstein
>> I do not think this is relevant to this discussion, unless they >> are changing RFC 5322s position on NULs. > >But, it seems like a question that IETF could clarify. I don't see how further clarification is necessary here? I mean, a 16MB single line in email is clearly a MUST NOT, but

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Paul Fox
ken wrote: > I'm sitting down to write or modify nmh code. Right now we have a lot > of code that assumes NUL-terminated C strings are safe to represent > email everywhere. My question is: is that a valid assumption? If > we are making that assumption, fine, let's be explicit and if someone

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-21 Thread Ken Hornstein
>> if a NUL appears in the header somewhere all bets are off. > >I think it would be fascinating to understand how that happened. Depending >on how the parse tree is done, it could be marginally bad, or catastrophic. > >I really would be amazed if this is seen in the wild. But its a big >network:

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-20 Thread George Michaelson
> if a NUL appears in the header somewhere all bets are off. I think it would be fascinating to understand how that happened. Depending on how the parse tree is done, it could be marginally bad, or catastrophic. I really would be amazed if this is seen in the wild. But its a big network: maybe

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-20 Thread Ken Hornstein
>I have received email with C-T-E set to binary. While I don't think it >was needed, I haven't checked closely. Facinating! I am curious: who/what sent this to you! Do you remember the MIME type? >> - Completely handle embedded NULs properly. This is arguably the most >> correct option but

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-20 Thread David Levine
David Malone wrote: > I wonder if it would be better to use fwrite instead of write, to > avoid mixing stdio and Posix-style output? (It would also avoid an > unbuffered write of 1 byte.) Good point. How about the attached? David diff --git a/uip/post.c b/uip/post.c index 820ed05b..a58e19a1

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-20 Thread David Malone
I wonder if it would be better to use fwrite instead of write, to avoid mixing stdio and Posix-style output? (It would also avoid an unbuffered write of 1 byte.) David.

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-20 Thread David Levine
I wrote: > This might not be much of a lift. m_getfld might handle NULs in bodies, > and the MIME parser comes close to handling them as well. m_getfld and the MIME parser do handle NULs. post(8) doesn't. I'd like to commit the attached patch. It uses write(2) instead of fprintf(3) and

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-19 Thread David Levine
Ken wrote: > But RFC 2045 says in ยง6.2: > >Thus there are no >circumstances in which the "binary" Content-Transfer-Encoding is >actually valid in Internet mail. > Also, I am not aware of "binary" being used as a C-T-E at all. I have received email with C-T-E set to binary. While I

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-18 Thread Steffen Nurpmeso
Ken Hornstein wrote in <20230219012125.2e48b1d7...@pb-smtp21.pobox.com>: |>Seems to me this is classifcation of attachment data, which will end up |>as octet-stream in that case. | |It's ... a little confusing! | |>For S-nail we more or less do what Heirloom mailx has done. | |Well, it

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-18 Thread Ken Hornstein
>Seems to me this is classifcation of attachment data, which will end up >as octet-stream in that case. It's ... a little confusing! >For S-nail we more or less do what Heirloom mailx has done. Well, it seems that in the message lexer if you encounter a NUL you just stop, from a_msg_scan():

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-18 Thread Steffen Nurpmeso
P.S.: Congratulations to your new release btw. I have written an OAuth helper in Python3 that suports OAuth for GMail, Microsoft, Yandex: curl -u moon:mars --basic -O https://git.sdaoden.eu/browse/s-toolbox.git/plain/oauth-helper.py It has a "manual" mode where it documents for GMail --

Re: (Not-so) hypothetical question: What to do about NULs?

2023-02-18 Thread Steffen Nurpmeso
Ken Hornstein wrote in <20230219001921.597ad1e0...@pb-smtp20.pobox.com>: ... |- mutt ... |[.]Internally mutt does |have an idea if the content contains a NUL (the CONTENT structure contains |a 'nulbin' member which contains the number of NUL bytes), but it's not |clear to me what happens

(Not-so) hypothetical question: What to do about NULs?

2023-02-18 Thread Ken Hornstein
I've been idly thinking about this for a while, and while the question might be simple I think it gets at some larger meta-issues we have never really agreed on how to resolve it properly. My question is, simply: What should happen when nmh encounters a NUL character (U+) in email? The rules