Re: Send international text with mail(1) - proposal and patches

2023-10-11 Thread Crystal Kolipe
On Thu, Oct 12, 2023 at 02:10:47AM +0200, Steffen Nurpmeso wrote: > Crystal Kolipe wrote in > : > |On Thu, Oct 12, 2023 at 12:36:48AM +0200, Steffen Nurpmeso wrote: > |> Non-7-bit clean headers need RFC 2047 (and/or RFC 2231) encoding. > | > |The use of MIME encoded words to encode header

Re: Send international text with mail(1) - proposal and patches

2023-10-11 Thread Steffen Nurpmeso
Crystal Kolipe wrote in : |On Thu, Oct 12, 2023 at 12:36:48AM +0200, Steffen Nurpmeso wrote: |> Non-7-bit clean headers need RFC 2047 (and/or RFC 2231) encoding. | |The use of MIME encoded words to encode header content is no longer |considered best practice. See, for example RFC 6532. Yes

Re: Send international text with mail(1) - proposal and patches

2023-10-11 Thread Crystal Kolipe
On Thu, Oct 12, 2023 at 12:36:48AM +0200, Steffen Nurpmeso wrote: > Non-7-bit clean headers need RFC 2047 (and/or RFC 2231) encoding. The use of MIME encoded words to encode header content is no longer considered best practice. See, for example RFC 6532. But as Omar said, let's get the basics

Re: Send international text with mail(1) - proposal and patches

2023-10-11 Thread Steffen Nurpmeso
Hello Omar. Omar Polo wrote in <2HJQ4VX5L4J1P.3G4A39B0RA6T7@venera>: ... |>MUAs always set appropriate MIME headers. RFC 2046 section 4.1.2 |>paragraph 8 also "strongly" recommends the explicit inclusion of a |>"charset" parameter even for us-ascii. So that really went me

Re: Send international text with mail(1) - proposal and patches

2023-10-11 Thread Omar Polo
Hello, Walter: I'm happy that you've been hacking on mail and at least in principle I think what you're doing makes sense; however, let's try to get one bit committed at a time. Let's start with the MIME needed for sending utf-8 messages. I've going through the various mail and I think it's

Re: Send international text with mail(1) - proposal and patches

2023-09-25 Thread wai
On Mon, 25 Sep 2023 21:31:08 +0200, Walter wrote: > Yours are the first technical, functional corrections I got about the > code. Thanks! Let's go back in time, then. I think that what you're > telling me can be done by simply replacing "break" for "return" in my > original function. Tell me

Re: Send international text with mail(1) - proposal and patches

2023-09-25 Thread Walter Alejandro Iglesias
On Mon, 25 Sep 2023 19:00:15 +0200, Hiltjo Posthuma wrote: > On Mon, Sep 25, 2023 at 03:13:03PM +0200, Walter Alejandro Iglesias wrote: > > This new version, when it detects invalid utf-8 in the body saves a > > dead.letter, prints the following message and exits. > > > > $ mail -s hello user <

Re: Send international text with mail(1) - proposal and patches

2023-09-25 Thread Hiltjo Posthuma
On Mon, Sep 25, 2023 at 03:13:03PM +0200, Walter Alejandro Iglesias wrote: > This new version, when it detects invalid utf-8 in the body saves a > dead.letter, prints the following message and exits. > > $ mail -s hello user < invalid_utf8.txt > Invalid or incomplete multibyte or wide

Re: Send international text with mail(1) - proposal and patches

2023-09-25 Thread Walter Alejandro Iglesias
This new version, when it detects invalid utf-8 in the body saves a dead.letter, prints the following message and exits. $ mail -s hello user < invalid_utf8.txt Invalid or incomplete multibyte or wide character . . . message not sent. Index: send.c

Re: Send international text with mail(1) - proposal and patches

2023-09-24 Thread Walter Alejandro Iglesias
On Sun, Sep 24, 2023 at 11:12:10AM -0300, Crystal Kolipe wrote: > On Sun, Sep 24, 2023 at 12:37:08PM +0200, Walter Alejandro Iglesias wrote: > > +static int > > +not_utf8(FILE *message, int len) > > +{ > > + int n, ulen; > > + unsigned char s[len]; > > Please re-read Omar's advice about large

Re: Send international text with mail(1) - proposal and patches

2023-09-24 Thread Crystal Kolipe
On Sun, Sep 24, 2023 at 12:37:08PM +0200, Walter Alejandro Iglesias wrote: > +static int > +not_utf8(FILE *message, int len) > +{ > + int n, ulen; > + unsigned char s[len]; Please re-read Omar's advice about large unbounded arrays.

Re: Send international text with mail(1) - proposal and patches

2023-09-24 Thread Walter Alejandro Iglesias
Hi Stefan, Do you like this? Index: send.c === RCS file: /cvs/src/usr.bin/mail/send.c,v retrieving revision 1.26 diff -u -p -r1.26 send.c --- send.c 8 Mar 2023 04:43:11 - 1.26 +++ send.c 24 Sep 2023 10:33:11

Re: Send international text with mail(1) - proposal and patches

2023-09-24 Thread Walter Alejandro Iglesias
On Sun, Sep 24, 2023 at 09:47:41AM +0200, Stefan Sperling wrote: > In the UTF-8 locale I can trigger an error message with your program > by sending the latin1 code for a-acute to stdin. I suppose your test > command didn't actually send latin1 to stdin for some reason? > > $ perl -e 'printf

Re: Send international text with mail(1) - proposal and patches

2023-09-24 Thread Stefan Sperling
On Sun, Sep 24, 2023 at 07:06:35AM +0200, Walter Alejandro Iglesias wrote: > Hi Ingo, > > On Thu, Sep 21, 2023 at 03:04:24PM +0200, Ingo Schwarze wrote: > > In general, the tool for checking the validity of UTF-8 strings > > is a simple loop around mblen(3) if you want to report the precise > >

Re: Send international text with mail(1) - proposal and patches

2023-09-23 Thread Walter Alejandro Iglesias
Hi Ingo, On Thu, Sep 21, 2023 at 03:04:24PM +0200, Ingo Schwarze wrote: > In general, the tool for checking the validity of UTF-8 strings > is a simple loop around mblen(3) if you want to report the precise > positions of errors found, or simply mbstowcs(3) with a NULL pwcs > argument if you are

Re: Send international text with mail(1) proposal and patches]

2023-09-23 Thread Crystal Kolipe
On Sat, Sep 23, 2023 at 12:10:41PM +0200, Walter Alejandro Iglesias wrote: > > On Thu, Sep 21, 2023 at 02:12:50PM +0200, Stefan Sperling wrote: > > > Your implementation lacks proper bounds checking. It accesses > > > s[i + 3] based purely on the contents of the input string, without > > >

Re: Send international text with mail(1) proposal and patches]

2023-09-23 Thread Walter Alejandro Iglesias
> On Thu, Sep 21, 2023 at 02:12:50PM +0200, Stefan Sperling wrote: > > Your implementation lacks proper bounds checking. It accesses > > s[i + 3] based purely on the contents of the input string, without > > checking whether len < i + 3. Entering the while (i != len) loop with You surely meant

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Walter Alejandro Iglesias
On Fri, Sep 22, 2023 at 06:57:24AM +0200, Walter Alejandro Iglesias wrote: > Below, a version without utf8 parser. I added a ASCII check for the > subject. The day will come when wscons support UTF-8, right? In the > meantime, just by being careful not to type iso-latin characters while > using

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Walter Alejandro Iglesias
Hi Ingo, On Thu, Sep 21, 2023 at 03:04:24PM +0200, Ingo Schwarze wrote: > As Stefan says, adding a hand-written UTF-8 parser to mail(1) is > clearly not acceptable. Below, a version without utf8 parser. I added a ASCII check for the subject. The day will come when wscons support UTF-8, right?

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Walter Alejandro Iglesias
On Thu, Sep 21, 2023 at 02:12:50PM +0200, Stefan Sperling wrote: > On Thu, Sep 21, 2023 at 01:25:01PM +0200, Walter Alejandro Iglesias wrote: > > I corrected many of the things you pointed me, but not all. The > > function I use to check utf8 is mine, I use it in a pair of little > > programs

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Ingo Schwarze
Hi, i fear this is getting a bit out of hand... Stefan Sperling wrote on Thu, Sep 21, 2023 at 02:12:50PM +0200: > On Thu, Sep 21, 2023 at 01:25:01PM +0200, Walter Alejandro Iglesias wrote: >> I corrected many of the things you pointed me, but not all. The >> function I use to check utf8 is

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Stefan Sperling
On Thu, Sep 21, 2023 at 01:25:01PM +0200, Walter Alejandro Iglesias wrote: > I corrected many of the things you pointed me, but not all. The > function I use to check utf8 is mine, I use it in a pair of little > programs which I've *hardly* checked for memory leacks. I know my > function looks

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Walter Alejandro Iglesias
On Thu, Sep 21, 2023 at 11:26:11AM +0200, Omar Polo wrote: > On 2023/09/21 10:55:47 +0200, Walter Alejandro Iglesias > wrote: > > On Wed, Sep 20, 2023 at 08:36:23PM +0200, Walter Alejandro Iglesias wrote: > > > On Wed, Sep 20, 2023 at 07:44:12PM +0200, Walter Alejandro Iglesias wrote: > > > >

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Crystal Kolipe
On Thu, Sep 21, 2023 at 11:26:11AM +0200, Omar Polo wrote: > Do we really need the two checks? WFIW, my original suggestion made off-list was about checking for 0xfe and 0xff only: Crystal wrote: > 0xfe and 0xff are invalid in utf-8. > > It _might_ be worth detecting them and in this case not

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Omar Polo
On 2023/09/21 10:55:47 +0200, Walter Alejandro Iglesias wrote: > On Wed, Sep 20, 2023 at 08:36:23PM +0200, Walter Alejandro Iglesias wrote: > > On Wed, Sep 20, 2023 at 07:44:12PM +0200, Walter Alejandro Iglesias wrote: > > > And this new idea simplifies all to this: > > > > In case anyone else

Re: Send international text with mail(1) - proposal and patches

2023-09-21 Thread Walter Alejandro Iglesias
On Wed, Sep 20, 2023 at 08:36:23PM +0200, Walter Alejandro Iglesias wrote: > On Wed, Sep 20, 2023 at 07:44:12PM +0200, Walter Alejandro Iglesias wrote: > > And this new idea simplifies all to this: > > In case anyone else is worried. Crystal Kolipe already pointed me out > that a better UTF-8

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Steffen Nurpmeso
Steffen Nurpmeso wrote in <20230920214009.w5mrf%stef...@sdaoden.eu>: |Ingo Schwarze wrote in | : | ... ||I just checked - even though i'm using the higer-level mutt(1) MUA ||most of the time and even though the shell i'm starting mutt(1) from ||has LC_CTYPE=C.UTF-8 set on that particular

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Steffen Nurpmeso
Ingo Schwarze wrote in : ... |I just checked - even though i'm using the higer-level mutt(1) MUA |most of the time and even though the shell i'm starting mutt(1) from |has LC_CTYPE=C.UTF-8 set on that particular machine, the last sixteen |mails i sent all contained the explicit header | |

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Kirill Miazine
Hi, Ingo • Ingo Schwarze [2023-09-20 13:55]: Hi Kirill, Kirill Miazine wrote on Wed, Sep 20, 2023 at 12:52:52PM +0200: you may not even need -m, and instead inspect LC_CTYPE environment variable and add appropriate headers for UTF-8. according to locale(1), LC_CTYPE may be set to indicate

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Walter Alejandro Iglesias
On Wed, Sep 20, 2023 at 07:44:12PM +0200, Walter Alejandro Iglesias wrote: > And this new idea simplifies all to this: In case anyone else is worried. Crystal Kolipe already pointed me out that a better UTF-8 checking is needed, I know, I'll get to that tomorrow.

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Walter Alejandro Iglesias
On Wed, Sep 20, 2023 at 06:13:10PM +0200, Walter Alejandro Iglesias wrote: > Now I was investigating exactly that :-) (like Mutt also does): to make > mail(1) automatically set the appropiate MIME headers when it detects > any utf8 characters in the body text. So, you don't like this idea? >

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Walter Alejandro Iglesias
On Wed, Sep 20, 2023 at 05:30:08PM +0200, Ingo Schwarze wrote: > Hi, > > i checked the following points: > > * Even though RFC 2049 section 2 bullet point 1 only *requires* >MIME-conformant MUAs to always write the header "MIME-Version: >1.0" - and mail(1) is most certainly not

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Ingo Schwarze
Hi, i checked the following points: * Even though RFC 2049 section 2 bullet point 1 only *requires* MIME-conformant MUAs to always write the header "MIME-Version: 1.0" - and mail(1) is most certainly not MIME-conformant - RFC 2049 section 2 bullet point 8 explicitly *recommends* that

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Ingo Schwarze
Hi Kirill, Kirill Miazine wrote on Wed, Sep 20, 2023 at 12:52:52PM +0200: > you may not even need -m, and instead inspect LC_CTYPE environment > variable and add appropriate headers for UTF-8. according to locale(1), > LC_CTYPE may be set to indicate UTF-8: > > If the value of LC_CTYPE ends

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Walter Alejandro Iglesias
On Wed, Sep 20, 2023 at 10:30:31AM +, Klemens Nanni wrote: > Except for mandoc(1) and other manuals where "utf8" is a literal keyword, > our manuals consistently use upper-case UTF-8 for what is an abbreviation, > so this should do as wlel. > > > .It Fl n > > Inhibits reading > > .Pa

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Kirill Miazine
• Walter Alejandro Iglesias [2023-09-20 12:21]: Hi Ingo, I did what you suggested me, I investigated a bit and you were right in that the MIME-Version header was necessary. This new set of patches add the following headers (hardcoded as you suggested me): MIME-Version: 1.0 Content-Type:

Re: Send international text with mail(1) - proposal and patches

2023-09-20 Thread Walter Alejandro Iglesias
Hi Ingo, I did what you suggested me, I investigated a bit and you were right in that the MIME-Version header was necessary. This new set of patches add the following headers (hardcoded as you suggested me): MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8

Re: Send international text with mail(1) - proposal and patches

2023-09-19 Thread Walter Alejandro Iglesias
On Tue, Sep 19, 2023 at 05:48:01PM +0200, Ingo Schwarze wrote: > Hi Walter, > > i did not look closely at the patch yet, and i did not dig for standards > documents, which one should almost certainly do before committing such > a patch unless one knows all the relevant standards by heart (which i

Re: Send international text with mail(1) - proposal and patches

2023-09-19 Thread Ingo Schwarze
Hi Walter, i did not look closely at the patch yet, and i did not dig for standards documents, which one should almost certainly do before committing such a patch unless one knows all the relevant standards by heart (which i do not), so i'm not saying this must be done differently, but instead i

Re: Send international text with mail(1) - proposal and patches

2023-09-19 Thread Walter Alejandro Iglesias
I'd forgotten that adding a "charset" specification to the Content-Type header is also needed. In the *new* set of patches below, besides I corrected some other errors, I added a '-y' option to specify utf-8 character set: $ mail -s Hello -e quoted-printable -y "text/plain;charset=utf-8" \

Send international text with mail(1) - proposal and patches

2023-09-19 Thread Walter Alejandro Iglesias
Hello everyone, Years ago I made a suggestion here about this, now that I am a little less ignorant in C I dared with the patches below, they add a '-e' option to mail(1) to pass a Content-Transfer-Encoding on the command line. First you convert the text in "message.txt" to quoted-printable or