Re: [Nmh-workers] Determining my draft folder

[Steffen Nurpmeso]

Ralph Corderoy  wrote:
 |I wrote:
 |> EXIT STATUS
 |>  mhparam returns the number of components that were not found.
 |...
 |> $ mhparam `yes | sed 256q`; echo $?
 |> 0
 |
 |I've just pushed to git.
 |
 |http://git.savannah.nongnu.org/cgit/nmh.git/commit/?id=e8eb3afba50cbec8\
 |d1aeabcf85a06084977d54cd
 |
 |Limit mhparam's exit status to 120 missing components.
 |Otherwise, 126 and 127 would clash with bash and zsh's use.
 |Higher than 127 would look like signals.  And 256 would wrap
 |to a falsely succesful zero.

Jörg Schilling pushed the 32-bit exit status for a future POSIX
version.  In how far this affects shells etc., … i haven't really
followed the discussion.

--steffen
___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [Nmh-workers] strncpy(3), die, die, die.

[Steffen Nurpmeso]

Hi.  Damn it's late...

Ralph Corderoy  wrote:
 |> Therefore i think a function like the Linux strscpy()
 |
 |That was a new one to me.  I found it, https://manned.org/strscpy.9
 |It's defined in the Linux kernel.
 |
 |ssize_t strscpy(char * dest, const char * src, size_t count);
 |
 |Does nothing if count is zero.  Otherwise, it's dest's size.
 |Copies src to dest, as much as fits, truncating, and ensuring it's NUL
 |terminated.
 |Returns -E2BIG if truncation occurs.

Yes, it is pretty new - i think it essentially is what strncpy()
should have been from the beginning.  Effectively

  FL ssize_t
  n_strscpy(char *dst, char const *src, size_t dstsize){
 ssize_t rv;
 NYD2_ENTER;

 if(LIKELY(dstsize > 0)){
rv = 0;
do{
   if((dst[rv] = src[rv]) == '\0')
  goto jleave;
   ++rv;
}while(--dstsize > 0);
dst[--rv] = '\0';
 }
  #ifdef HAVE_DEVEL
 else
assert(dstsize > 0);
  #endif
 rv = -1;
  jleave:
 NYD2_LEAVE;
 return rv;
  }

You know, -E2BIG and POSIX ssize_t is allowed to be invalid, but
i don't get the fun behind that.  ssize_t is not ISO C, that is
anything else but fun, in my opinion.
The Linux kernel guys have used their usual optimization.  (I
really thought it is the same that i have seen around Y2K for
generic strlen().  Or was it GNU LibC strlen()?  But that one.)
Also ~U (!U)ing this (late!):

 |> i use asprintf() for this kind of thing.
 |
 |It's nice, but it might do the formatting work twice, and the return
 |value needs checking, not just for "out of memory" errors, the char** is
 |not guaranteed to be NULL on error with GNU, and that checking conflicts
 |with the "minimal call-site change" that's my aim.

In my opinion that is really terrible.  Indeed i am very, very
prowd of my context idea, especially so now that i have been
pointed to Plan9 and i have seen that the really extraordinary
guys had the same idea.  I.e., you have a format context object
and call in, from I/O stream, dynamic string, stack buffer,
whatever, and it does as much as it can, and gives back a status.
Then you can feed in more buffer and continue where it stopped
(not restart) if you want to.  The standard I/O goes the very
opposite way, you feed in a string and at the back of the scene it
dispatches readily prepared data, but that interface not
standardized.  That is _so_ terrible.  Always hated it.
Good night, and a nice Sunday!
Ciao

--steffen

___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [Nmh-workers] strncpy(3), die, die, die.

[Steffen Nurpmeso]

Paul Vixie  wrote:
 |Ralph Corderoy wrote:
 |> Perhaps a complainant could be told of the secret $NMHNOBARF to stop
 |> TRUNCCPY from aborting?  Though it would still complain for the first N
 |> goes?
 |
 |i think the moment that the state of the program becomes undefined, you
 |should abort.
 |
 |malloc and asprintf helpfully return a useless value (NULL) if they
 |can't fit your result into a new heap blob.
 |
 |snprintf, strncpy, and strlcpy do not.

I disagree, for the former and the latter.

 |the right thing to create on overflow is an empty string. if the caller
 |doesn't check the return value, they're going to wonder where that empty
 |string came from. this would teach callers to check return values.
 |
 |returning the front half of the source string is bad. and while
 |returning it non-\0-terminated is worse, neither is acceptable.

But in effect i think it was always an error to do such
programming, because in multibyte environments truncating a string
isn't at all easy.

I think the former and latter of the above have the problem that
they return useless information: the size that would be necessary
to store the result in a non-truncated form.  If that information
would be collected regulary using the above functions i would
think that this is really wasteful software that gives a s..t on
the necessity of, e.g., atomic plants etc.  I think it is this
kind of mental direction that brings us all down.

I think the problem of strncpy() is that it doesn't NUL terminate
in the one, or zero-pads in the other case, which are both things
that i don't want.  Therefore i think a function like the Linux
strscpy() is something than can be used in, e.g., all US-ASCII
/ Deutsch / any single-byte locale servers or other programs,
where truncating a log entry etc. is to be performed (cheaply).

Otherwise i would claim that usage of such a function is likely an
error, and should be replaced by dynamic string objects.
E.g., i still have four use cases of strscpy() in the MUA
i maintain, and they are all wrong.

 |i once received a thousands-of-lines-long patch to bind8 to make it use
 |snprintf and strlcpy. i rejected it, and replaced every caller whose
 |starting conditions were not obvious from simple inspection with an "if"
 |statement that crashed out of the current operation if the resulting
 |string would not fit my assumptions.
 |
 |replacing overrun with truncation is not a big enough improvement to
 |justify touching the code at all.

--steffen

___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [Nmh-workers] Changes to forw(1)

[Steffen Nurpmeso]

Hello Ralph,

this likely goes to you in private only (via News)

Ralph Corderoy  wrote:
 |kre wrote:
 |> Rather, I usually delete [X-Mailer] as I consider it no-one else's
 |> business which software I use
 |
 |Yes, I'd want to turn it off from that point of view too.  I use mail(1)
 |for short internal emails, and here it's provided by package s-nail
 |which had a series of different core-dump bugs that I triggered.  It
 |quite possibly has buffer overflows  on receiving and rendering emails

Granted.  This is indeed not true.

 |too, and puts out `User-Agent: mail v14.8.12' into every email, like
 |pinning a target to the user's back.  :-)

This is a good suggestion, i should have introduced *user-agent*
just like mutt(1) has, instead of extending *stealthmua* à la "set
stealthmua=noagent" or -Sstealthmua=noagent!
Ciao!

--steffen

___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [Nmh-workers] Request Deprecation of mts.conf's mmdelim1 and mmdelim2.

[Steffen Nurpmeso]

Robert Elz  wrote:
  ...
 |If the mbox encoding format had been properly designed, rather than just
 |a "we need some way to fix the problem that a line starting 'From' in
 |a message acts like a separator" (which was a real issue/bug in early
 |implementations of it) this issue wouldn't arise.

It seems it has been simply be carried along from original Unix
mail storage, and never has been properly adjusted thereafter.
POSIX also standardized this loose format which was in use since
that beginning.  RFC 4155 defined a more proper format.

In the end MBOX is just a database format, the MUA i maintain uses
a MIME content-encoding to ensure the assertion of this format is
not contradicted, but we do not yet re-encode messages like that
when copying over, for that we have to perform proper From_
quoting as necessary, then.  This is a huge pile of cr.p.  Still,
using RFC 4155 rules for one lowers the possibilities for database
format clashes, and may give surprises due to different detection
of message boundaries, our upcoming version will warn when opening
mailboxes where this could be a problem.

 |There was one attempt to use (essentially, though as I recall, not quite
 |implemented that way) out of band data for mail collection files - that
 |is, adding a header and length field (much like tar format, or cpio, etc,

Jamie Zawinski of Netscape+ on that[1] is pretty famous:

  [1] https://www.jwz.org/doc/content-length.html

 |but much simpler) but which was a spectacular failure, and hated much more
 |than '>From' (which is mostly an annoyance, though it does screw integrity

My MUA strips those because it doesn't manage them, unless the
variable *keep-content-length* is set.  I think i have to obsolete
this, and at some future time either silently drop or fully
support them.  But they are redundant if messages are
content-encoded, and in times of digital signatures etc. you need
to make sure that user input is identical to output anyway, so
proper content-encoding is the right thing to do.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [Nmh-workers] Request Deprecation of mts.conf's mmdelim1 and mmdelim2.

[Steffen Nurpmeso]

Ken Hornstein  wrote:
 |>It seems it has been simply be carried along from original Unix
 |>mail storage, and never has been properly adjusted thereafter.
 |>POSIX also standardized this loose format which was in use since
 |>that beginning.  RFC 4155 defined a more proper format.
 |
 |I think there are two things that are being conflated here: the
 |various "standards" of the mbox format, and nmh's use of it.
 |
 |Since nmh doesn't use mbox as a mail store, things like RFC 4155 aren't
 |really relevant; we don't deal with mbox files except for two specific
 |tools.  So our goal here is to deal with existing mail DROPS (I use
 |that term to specify a place where external tools store mail where nmh
 |can read it).

Well, using RFC 4155 parse rules for From_ lines can lead to
different message boundary detection.

 |Of course, the more I dig into it the more fun I find.  For example:
 |
 | https://en.wikipedia.org/wiki/Mbox
 |
 |Which suggests that SOME mbox formats do perform reversible
 |From-munging.  Urrrk.

I removed that from the MUA i maintain, because you never know for
sure: for that you would need to scan the entire message first,
check if several From_ lines occur and have been quoted alike,
before you start to remove what you think is a superficial From_
quote.  And then ezml i think it was (what unicode.org had before
the switch to i think mailman) simply placed a space in the first
column...  Any non truly-reversible change changes the original
content, applying a MIME content-encoding is standardized, there
you go.

 |This suggests to me that a "next-gen" maildrop parser should be prepared
 |to handle what Wikipedia calls "mboxo" and "mboxrd" format.  A web page
 |linked to on that page suggests that on Linux Content-Length variants
 |(mboxcl and mboxcl2) are more common on Linux, but I am skeptical that
 |is true.

mutt actively manages these header lines, at least.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

___
Nmh-workers mailing list
Nmh-workers@nongnu.org
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] multipart/related and -prefer.

[Steffen Nurpmeso]

Ralph Corderoy wrote in <20180731135013.d6e8b21...@orac.inputplus.co.uk>:
  ...
 |I recently read this `Apple Mail' email with mhshow, read the
 |plain/text, and it wasn't apparent to me there were images too.
 ...  
 |$ mhlist -nov -prefer text/plain
 | msg part  type/subtype  size description
 |32753   multipart/alternative6185K
 |  1 text/plain 134
 |  2 multipart/related6184K
 |  2.1   text/html  721
 |  2.2   image/jpeg   2176K
 |  2.3   image/jpeg   2401K
 |$ 
 |
 |Typically, the text/plain and text/html are children of the same
 |multipart/alternative, but here Apple thinks text/plain means images
 |aren't required.  Perhaps revenge for my never having purchased an Apple
 |product.

I would not blame Apple, i have seen this from mailers in the
Microsoft world a lot over three years ago, but i have not yet
found any time to complete something first committed on 2015-09-19

FIXME [mimepipe.3] *mime-alternative-favour-rich*: honour /related..

Especially in the Microsoft world we can see /alternative
messages which have a whole bunch of /alternative parts
collected in a single /related part, i.e.,

  alternative
text/plain
multipart/related
  whole bunch of parts, e.g., HTML, images, etc.

We want to print the entire /related part when used as
an alternative.

Recursive descendent MIME parsers really suck.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Unnecessary dependency on vi???

[Steffen Nurpmeso]

Ralph Corderoy  wrote:
 |>> so a program like mail would offer two escapes (~e vs.  ~v) to let
 |>> yo invoke either.
 |>
 |> So ... I guess programs would look at the terminal and if your speed
 |> was 9600 baud or greater, you'd use VISUAL, and if it was slower you'd
 |> use EDITOR?
 |
 |No, AFAIK it was always the user's choice.  mail(1) had the `~e' escape
 |and then added a `~v' one, with VISUAL and EDITOR environment variables
 |echoing the cpp(1) macro names of the default values.  Kurt Shoens,
 |k...@ucbvax.berkeley.edu, is down as the author in
 |BSD-1-253-gc145e9e0ab5 of
 |https://github.com/dspinellis/unix-history-repo.

BSD Mail had both of ~v and ~e from the very start.  I know of no
known released file which acted otherwise.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Unnecessary dependency on vi???

[Steffen Nurpmeso]

Bakul Shah <ba...@bitblocks.com> wrote:
 |On Tue, 20 Mar 2018 23:57:28 +0700 Robert Elz <k...@munnari.oz.au> wrote:
 |Robert Elz writes:
 |> Date:Tue, 20 Mar 2018 15:43:37 +0100
 |> From:    Steffen Nurpmeso <stef...@sdaoden.eu>
 |> Message-ID:  <20180320144337.zm2ro%stef...@sdaoden.eu>
 |> 
 |>| BSD Mail had both of ~v and ~e from the very start.  I know of no
 |>| known released file which acted otherwise.
 |> 
 |> Including in the first BSD distribution tape (1BSD) before vi existed \
 |> (also 
 |> before csh existed, but that's unrelated...) ??
 |
 |I looked at 1bsd and 2bsd distributions. I don't see Mail in
 |1bsd but it is included in 2bsd and uses VISUAL for ~v and
 |EDITOR for ~e.

No, it cannot be in 1BSD if we trust

  /*
   * Mail -- a mail program
   *
   * Author: Kurt Shoens (UCB) March 25, 1978
   */

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Unnecessary dependency on vi???

[Steffen Nurpmeso]

Hello Ralph.

Ralph Corderoy  wrote:
 |>> mail(1) had the `~e' escape and then added a `~v' one, with VISUAL
 |>> and EDITOR environment variables echoing the cpp(1) macro names of
 |>> the default values.  Kurt Shoens, k...@ucbvax.berkeley.edu, is down
 |>> as the author in BSD-1-253-gc145e9e0ab5 of
 |>> https://github.com/dspinellis/unix-history-repo.
 |>
 |> BSD Mail had both of ~v and ~e from the very start.  I know of no
 |> known released file which acted otherwise.
 |
 |But peering at doc/Mail/mail3.nr in BSD-1-3-gfc8c50acc08, so just after
 |BSD 1 was cut, I see it documents all the tilde escapes and has `~e' but
 |no `~v'.
 |https://github.com/dspinellis/unix-history-repo/blob/fc8c50acc0870bf28753d35\
 |08770428682e915bb/doc/Mail/mail3.nr
 |
 |By the time of BSD-1-54-ge684660a6a2, src/Mail/Mail.help.~ lists both.
 |https://github.com/dspinellis/unix-history-repo/blob/e684660a6a291c1e4672912\
 |bc1b80ffb00934623/src/Mail/Mail.help.%7E
 |
 |So although the released code had both, I think it's likely that `~e'
 |was there on its own, and then `~v' added as ex's vi mode came along.
 |I also noticed that Mail's string option was at one point `EDITOR' for one
 |and `VISEDITOR' for the other;  also suggestive that one came first
 |rather than both together.

Ok, i do not have Spinellis repo locally (yet), it is too big.
(How large is it in the end, Ralph?)

Looking at github i see at the same commit [1], and just in case
i do that right, that both of the `edit' and `visual' commands are
already available, so maybe ~v had only been forgotten by that
time?

  It is often useful to be able to invoke one of two editors,
  based on the type of terminal one is using.  To invoke
  a display oriented editor, you can use the
  .b visual
  command.  The operation of the
  .b visual
  command is otherwise identical to that of the
  .b edit
  command.

  [1] 
https://github.com/dspinellis/unix-history-repo/blob/fc8c50acc0870bf28753d3508770428682e915bb/doc/Mail/mail6.nr

Really a shame i do not have Spinellis repo yet.  In the repo
i only have history back to 2BSD...
Ciao,

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Unnecessary dependency on vi???

[Steffen Nurpmeso]

Bakul Shah  wrote:
 |On Mar 20, 2018, at 5:04 PM, Ralph Corderoy  wrote:
 |>>> mail(1) had the `~e' escape and then added a `~v' one, with VISUAL
 |>>> and EDITOR environment variables echoing the cpp(1) macro names of
 |>>> the default values.  Kurt Shoens, k...@ucbvax.berkeley.edu, is down
 |>>> as the author in BSD-1-253-gc145e9e0ab5 of
 |>>> https://github.com/dspinellis/unix-history-repo.
 |>> 
 |>> BSD Mail had both of ~v and ~e from the very start.  I know of no
 |>> known released file which acted otherwise.
 |> 
 |> But peering at doc/Mail/mail3.nr in BSD-1-3-gfc8c50acc08, so just after
 |> BSD 1 was cut, I see it documents all the tilde escapes and has `~e' but
 |> no `~v'.
 |> https://github.com/dspinellis/unix-history-repo/blob/fc8c50acc0870bf28753d\
 |> 3508770428682e915bb/doc/Mail/mail3.nr
 |
 |
 |This is a doc. bug! ~v is mentioned in mail7.nr in the
 |same fc8c50acc0 "commit". This has April 19, 1979 date.
 |The same as the 2bsd date on TUHS unix-archives. See my
 |previous message. ~v was already in!

It was in in 2BSD, yes.

 |> By the time of BSD-1-54-ge684660a6a2, src/Mail/Mail.help.~ lists both.
 |> https://github.com/dspinellis/unix-history-repo/blob/e684660a6a291c1e46729\
 |> 12bc1b80ffb00934623/src/Mail/Mail.help.%7E
 |> 
 |> So although the released code had both, I think it's likely that `~e'
 |> was there on its own, and then `~v' added as ex's vi mode came along.
 |> I also noticed that Mail's string option was at one point `EDITOR' for one
 |> and `VISEDITOR' for the other;  also suggestive that one came first
 |> rather than both together.
 |
 |This is in mail7.nr but looking at the sources, it is 'VISUAL'
 |so I suspect this is another doc. bug (both bugs are also in
 |doc/Mail/mail{3,7}.nr in the 2bsd dist. on TUHS).
 |

Only EDITOR and VISUAL in 2BSD, yes.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Unnecessary dependency on vi???

[Steffen Nurpmeso]

Ralph Corderoy  wrote:
 |> Ok, i do not have Spinellis repo locally (yet), it is too big.
 |> (How large is it in the end, Ralph?)
 |
 |1.5 GiB.  Too large to pull home with ADSL.  I've a VM out on the
 |Internet that has fast connectivity and I copied it there.

Interesting idea... but is has to wait.  I remember a message of
a github staff member who was sympathetic to the community because
the average member used far less space than github would have
offered.  Must have been 2011 or something.  I.e., different to
some other formerly large player i expect Spinellis repo to be
available until i finally upgrade my little one...

 |But don't worry about replying after a delay.  I've nmh emails that are
 |months old that I still intend to process.  :-)

Inbox first, yes.  Of course!

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Unnecessary dependency on vi???

[Steffen Nurpmeso]

Robert Elz <k...@munnari.oz.au> wrote:
 |Date:Tue, 20 Mar 2018 15:43:37 +0100
 |From:    Steffen Nurpmeso <stef...@sdaoden.eu>
 |Message-ID:  <20180320144337.zm2ro%stef...@sdaoden.eu>
 |
 || BSD Mail had both of ~v and ~e from the very start.  I know of no
 || known released file which acted otherwise.
 |
 |Including in the first BSD distribution tape (1BSD) before vi existed (also 
 |before csh existed, but that's unrelated...) ??
 |
 |This also predates the use of SCCS (or any other revision control system)
 |at UCB.

Now you got me.  No.
(Things that happened in the past only happened in your mind.
So, forget your mind, and you'll be: free.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Making OpenSSL 1.0.2 minimum version

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190627175008.639987b...@pb-smtp21.pobox.com>:
 |>Thinking about it, the "ext" in SSL_set_tlsext_host_name
 |>could appear strange in five years from now.
 |
 |As opposed to the REST of the OpenSSL API? :-)

..seen that way..  But the problem is real:

  #?0|kent:$ grep -ri deprecated /usr/include/openssl/|wc -l
  85

And this lists prominent things like and as new as TLSv1_2_XY().

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Making OpenSSL 1.0.2 minimum version

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190627171410.ea24e7b...@pb-smtp21.pobox.com>:
 |>I use that protected via
 |>
 |>  #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME
 |
 |I did see that ... but I also was worried that since OpenSSL makes no
 |guarantees that this define will stick around in the future, depending
 |on that may come back to bite me.  I'd rather simply just put it in
 |unconditionally and force everyone to be using 1.0.0 or newer.

Fair enough.  Though i am afraid that regarding OpenSSL bit rot
will have to be expected; the _CTRL_ series looked the most stable
to me.  Thinking about it, the "ext" in SSL_set_tlsext_host_name
could appear strange in five years from now.  Btw. i was lazy and
simply call this function, even if SSLv3 was still around by
then (more than today): OpenSSL and derivates do not perform any
checks, it is just that the hostname set will be used for SNI if
possible, and not otherwise.  Unlikely this has changed.  (Despite
that noone uses SSLv3 no more.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Formatting HTML to Text: netrik.

[Steffen Nurpmeso]

Ralph Corderoy wrote in <20190627125114.70a3921...@orac.inputplus.co.uk>:
 |Revisiting once again the issue of nice text from horrible HTML emails,
 |I found another textual web browser, like lynx(1), etc., but
 |http://netrik.sourceforge.net/ uses colour in its --dump output.
 |It doesn't list the URL destinations though, e.g. the noisy `[42]' that
 |links prefixes to the anchor's text.  I also haven't looked into what
 |remote accesses it may make.

You could try to take out the very simple and extremely primitive
HTML filter code of the MUA i maintain and embed it into nmh.  It
only uses standard POSIX C interfaces, so it should not be too
hard.  I do not use anything else for the few HTML things i get,
i have never seen it fail.  (I have test mails around which count
as terror of a pathological maniac.  The only one where i can see
some garbage is actually from the german computer magazine c't,
and it looks like

  [-- #1.2 1007/75296 text/html, quoted-printable, utf-8 --]

  96

  Silex: Neue Malware legt schlecht gesicherte Geräte im Internet of Things 
still -- Die Malware Silex 
kapert mit Default-Credentials IoT-Geräte, um sie
  lahmzulegen. Ihr 14-jähriger Entwickler handelt offenbar aus Spaß. [%p(2,2)]» 
https://www.heise.de/security/meldung/Silex-Neue-Malware-legt-schlecht-gesicherte-Geraete-im-Internet-of-Things-still-
  4455677.html[%hr(=)]

Wherever "96", "%p(2,2)" and "%hr(=)" come from and whatever they
are, i never looked into this.  I only ever see this from these
guys.  But WWF, Naturschutzbund and Conversation work for years,
as well as some german shops, and Change.org and a lot of other
mails in the HTML test box do, too.
Guaranteed no loading of external data, anyway.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] Making OpenSSL 1.0.2 minimum version

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190627150420.4ff107a...@pb-smtp21.pobox.com>:
 |Everyone,
 |
 |When researching the issue Michael Richardson brought up today, it make
 |me realize we really should be calling SSL_set_tlsext_host_name() so we
 |send the TLS extension "server name indicator".  Which is easy, it's
 |literally one line of code.  But that makes me ask a larger question: we
 |have some autoconf goo to support older libraries (pre OpenSSL 1.0.2)
 |that didn't support the function X509_VERIFY_PARAM_set1_host(), and I
 |lack the energy to research if SSL_set_tlsext_host_name() exists in
 |pre-1.0.2 OpenSSL.  I think at this point we should consider OpenSSL
 |1.0.2 the minimum supported version of OpenSSL for nmh.  This would
 |guarantee we are doing TLS 1.2 everywhere and clean up some #ifdefs.
 |Objections?

I use that protected via

  #ifdef SSL_CTRL_SET_TLSEXT_HOSTNAME

which seems to work everywhere i tried.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] logging outgoing messages

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190710152824.2d9b961...@pb-smtp21.pobox.com>:
  ...
 |all of the time?  Secondly ... I am seeing more and more authentication
 |methods that require keeping some kind of state and possibly user
 |interaction in the MUA (GSSAPI and XOAUTH2 are two examples that I have
 |personally encountered), and that makes doing authentication in the MTA

I could not disagree more, and if its for political reasons;
i think that today with TLS plain passwords are all you need,
other cruft should leave codebases as soon as possible.  The only
exception comes with the availability of a system like Kerberos,
which can provide you local tickets, with timeouts etc. as
requested, shared in between multiple applications in a secure
way.  I once had "kdestroy -A" in my shell logout file, today
i would hook that into my on-lid-close script.

Unfortunately i am too stupid to do the real thing and use GELI on
FreeBSD aka dm-crypt/LUKS on Linux, ie block level encryption, but
even i have an encfs directory which serves my config files, and
one encfs loaded once a week which stores the keys.  The former
includes a PGP encrypted .netrc-style file, which holds all the
credentials for Google and my S/MIME keys (my MUA supports
"pseudo-hosts" like USER@HOST.smime-cert-key, .smime-cert-cert and
.smime-include-certs), and becomes decrypted on the fly.  Of
course my MUA is still primitive and kees that decrypted stuff in
clear, neither does it mprotect() the region nor zeroes that after
use.  I do not use suspend-to-disk, but still.  And it would be
better with encfs2, but that will not happen i guess.

 |very problematic.  I think the days of embedded plaintext passwords in
 |your MTA configuration file are slowly coming to an end.

Some kind of shared TLS private key and password service that
daemons can use to load such, before they start their privilege-
separated childs which only have the readily prepared sessions.
And to be unlocked with a Yubikey first.  (And with an option to
implant that under the skin of an administrators living flesh.)
Brave new world.

 |Like I said in my previous email ... we'll continue to support that.
 |But I can't recommend it to the average nmh user.
 |
 |--Ken
 |
 |-- 
 |nmh-workers
 |https://lists.nongnu.org/mailman/listinfo/nmh-workers
 --End of <20190710152824.2d9b961...@pb-smtp21.pobox.com>

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] FSF is changing Mailman list settings unless you opt out (fwd)

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190926163645.c801a82...@pb-smtp20.pobox.com>:
 |Everyone,
 |
 |I received this email, and I wanted to pass it along.  The executive
 |summary is: in the near future subject lines to nmh-workers will no
 |longer be prefixed with "[nmh-workers]" and there won't be a footer
 |at the end of the message anyone saying that this is the nmh-workers
 |mailing list.  You can read the details in the message for the complete
 |technical reasons why this is happening; the other option is to do
 |what is called "Munge From" and I personally think this is 100x worse
 |(I am on a Yahoo mailing list where this is done and I hate it).  If
 |people think we should switch to "Munge From" for this mailing list,
 |then please make your case here.

Yuck.  As a purely rhetorical note, do they have a plan to upgrade
from the TLS 1.0 they use.  (And i hope this does not qualify as
sexual harassment.  It is not!  I eat at home, like the Beatles.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] FSF is changing Mailman list settings unless you opt out (fwd)

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190926174635.3b44384...@pb-smtp20.pobox.com>:
 |>Yuck.  As a purely rhetorical note, do they have a plan to upgrade
 |>from the TLS 1.0 they use.  (And i hope this does not qualify as
 |>sexual harassment.  It is not!  I eat at home, like the Beatles.)
 |
 |I ... do not know about the TLS 1.0 issues, nor do I see how it's relevant
 |to this discussion.

I am sorry, i was in galop and you had to suffer the consequences.
It was also not meant to address you as "you", it is just that
i always hit "r" and if there is no reply-to: or mail-followup-to:
then the list is not the sole receiver.

I am not a cryptographer therefore i also do not know about TLS
1.0 issues, except .. that diediedie IETF draft, that the money
changers deprecated it to June 2018 at latest, and that the big
companies deprecate it (and the different/newer 1.1) in
.. spring (?) next year.  And that i really would like to slim my
vserver ssl/tls conf.  (eggs.gnu.org is the _only_ mail service
that i know that uses TLS1.0:DHE_RSA_AES_256_CBC_SHA1;
i accidentally stumbled over this a few months ago, when looking
into my archives.)

And it is entirely unrelated to this thread of course.
I personally feel sad because of the direction all this goes to.
That From: rewriting is just sick, it makes me sick.  Thank you.
RFC 4871 on DKIM says at least

   A common practice among systems that are primarily redistributors of
   mail is to add a Sender header field to the message, to identify the
   address being used to sign the message.  This practice will remove
   any preexisting Sender header field as required by [RFC2822].  The
   forwarder applies a new DKIM-Signature header field with the
   signature, public key, and related information of the forwarder.

whereas the Yahoo! only RFC 7489 says

   It has been suggested in several message authentication efforts that
   the Sender header field be checked for an identifier of interest, as
   the standards indicate this as the proper way to indicate a
   re-mailing of content such as through a mailing list.

   1.  The main user protection approach is to be concerned with what
   the user sees when a message is rendered.  There is no consistent
   behavior among MUAs regarding what to do with the content of the
   Sender field, if present.  Accordingly, supporting checking of
   the end user might never actually see, which can create a vector
   for attack against end users by simply forging a Sender field
   containing some identifier that DMARC will like.

For the MUA i maintain they at least can when they want.
What the  is that?  People are too stupid to get this
additional field right (look who they are voting!), so lets just
not even consider this.
This goes in line with the web browser community, they also do not
get it right and do not show TLS status, content blocking, Unicode
related lookalike thingies, or any such stuff.  No, not me,
Yahoo!, this is too exhausting, and i do not have any control over
it!!

   2.  Although it is certainly true that this is what the Sender field
   is for, its use in this way is also unreliable, making it a poor
   candidate for inclusion in the DMARC evaluation algorithm.

They break a field already present in RFC 822 from 1982.  That is
certainly true.  They should just have followed the RFCs and maybe
adjusted From: to also include the list address, then resign it
(maybe), moving the original author to Sender:.  Maybe.  But hey
the job is done, maybe they got a bonus.  Sounds bitter.  Baeh.

Good night from Germany.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] FSF is changing Mailman list settings unless you opt out (fwd)

[Steffen Nurpmeso]

hymie! wrote in <20190927142152.ga21...@alfred.local.net>:
 |On Fri, Sep 27, 2019 at 02:54:14AM -0700, Paul Vixie wrote:
 |> Ken Hornstein wrote on 2019-09-26 09:36:
 |>> I received this email, and I wanted to pass it along.  The executive
 |>> summary is: in the near future subject lines to nmh-workers will no
 |>> longer be prefixed with "[nmh-workers]" and there won't be a footer
 |>> at the end of the message anyone saying that this is the nmh-workers
 |>> mailing list.
 |> 
 |> yahoo is way off the reservation with this. they aren't growing fast \
 |> enough
 |> for me to care which of their users can't join mailing lists i operate \
 |> -- so
 |> my first mover advantage dominates the outcome.
 | 
 |Unfortunately, Yahoo isn't the only culrpit. More and more servers are
 |honoring DMARC.  I, for example, keep my email on my own server at home,
 |but because my ISP blocks port 25, I have to hire a third party to
 |receive and re-send my email for me, both incoming and outgoing.  I'm
 |(mostly) at their mercy for things like spam filters and DMARC, and
 |(at least right now) I'm not in a position to find a new company
 |if I decide I'm not crazy about their policies.
 |
 |I appreciate your opinion, but remember to look at it from the other
 |side -- what is the point of a mailing list if you're the only person
 |left who can access it?

But that makes me wonder a bit: how much does that cost?
Isn't it cheaper to rent the smallest possible vserver, and simply
mirror the local server, then adjust one line of configuration, or
two?

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: [nmh-workers] FSF is changing Mailman list settings unless you opt out (fwd)

[Steffen Nurpmeso]

Ken Hornstein wrote in <20190926231658.6947788...@pb-smtp20.pobox.com>:
 |>And it is entirely unrelated to this thread of course.
 |
 |I mean, at least we're in agreement there :-/  And I'm not even sure
 |WHY you care about what version of TLS that eggs.gnu.org supports, since
 |it is handling messages for public mailing lists.

That is a rabbit starter.  You know, if it would be me then TLS
would not be needed at all.  That entire complex bores me to
paralysis.  But there are bad people, everywhere, so this stacks
up from and to governments from and to industry and even from and
to science.  Maybe that is why people then enlighten their ripped
backsides by publishing their (maybe) deep(est) inside on some
public maybe social media, including photos.  I don't know.

 |>I personally feel sad because of the direction all this goes to.
 |
 |Sigh.  Well, if you want to just scream at the void, ok ... I can
 |understand that (I wish you wouldn't scream on nmh-workers, personally).
 |But ... DKIM/SPF/DMARC all exist because of spam.  While I don't agree

Hm.

 |with Yahoo's solution and if we had to do it all over we'd deal with
 |things very differently, I understand and can sympathize with the
 |decisions they have made.

Oh.  I can go up the wall on occasions, really.  That is certainly
true.  I cannot, absolutely not.  Maybe they should have invented
or reused the MIME types which exist for signed and encrypted data
instead.  (I know of two, S/MIME and OpenPGP.)

  ...
 --End of <20190926231658.6947788...@pb-smtp20.pobox.com>

Valdis Klētnieks wrote in <73912.1569546833@turing-police>:
 |On Thu, 26 Sep 2019 12:36:41 -0400, Ken Hornstein said:
 |> I received this email, and I wanted to pass it along.  The executive
 |> summary is: in the near future subject lines to nmh-workers will no
 |> longer be prefixed with "[nmh-workers]" and there won't be a footer
 |> at the end of the message anyone saying that this is the nmh-workers
 |
 |OK by me., my procmailrc already uses '*^(To:|cc:).*nmh-workers@nongnu.org'
 |so it won't bother my set-up.

I for one archive this list among others in a box called "track".
It is nice to see what is what from a glance, like, e.g.,

  Re: [art] Auto-configuring Email Clients via WebFinger
  Re: [Resolverless-dns] Paper on Resolver-less DNS

Since many discussions or topics actually spread several lists,
not only on IETF but everywhere, it is nice to see them
altogether (at least in date sorted mode).  For example in
a heritage site something comes up and reminds someone of
something else, which then leads to a thread in a different
heritage side.  I store those in one box, and then you get the
brainwaves a bit.  And at a glance.  I like this.

  ...
 --End of <73912.1569546833@turing-police>

Andy Bradford wrote in <20190926210904.82686.qmail@angmar.bradfordfamily\
.org>:
 |Thus said Ken Hornstein on Thu, 26 Sep 2019 12:36:41 -0400:
 |> You can  read the details  in the  message for the  complete technical
 |> reasons  why this  is happening;  the other  option is  to do  what is
 |> called "Munge From" and I personally think this is 100x worse (I am on
 |> a Yahoo mailing list where this is done and I hate it).
 |
 |I won't  mind if the  Subject stops being  mangled (I use  the list-post
 |header for filtering). I also won't mind if the body stops being mangled
 |with additional MIME parts; the same information is already available in
 |the  list-(un)?subscribe  headers) and  I'm  of  the opinion  that  list
 |control mechanisms  belong in headers.  Besides, how often  does someone
 |accidentally get subscribed to a mailing list anymore?

I agree with the headers, and i do not inject footers in the ML
myself.  That has much to do with that i am or was subscribed to
MLs where, if MIME mails come in, the ML software was not smart
enough to inject the footer in the text part, but simply added
a MIME part with nothing but the footer, and that was an outcome
that i really disliked.  (Have not seen that behaviour in a while,
consciously at least.)

 |I don't think however, that "Munge From" sounds like a good idea.

Yeah.

  ...
 --End of <20190926210904.82686.qm...@angmar.bradfordfamily.org>

Ralph Corderoy wrote in <20190927085932.d3b581f...@orac.inputplus.co.uk>:
 |> the other option is to do what is called "Munge From"
 |
 |That's one other option.
 |
 |Note, nmh-workers has been configured with
 |dmarc_moderation_action="Munge From" for ages and no one here has
 |grumbled.  It typically only munges Froms from Yahoo!  I expect the
 |other two nmh lists match this as I went through all their options on
 |becoming list admin.
 |
 |Another option supported by this Mailman is, in the Yahoo! case, to
 |
 |Wrap Message
 |Just wrap the message in an outer message with the From: header
 |containing the list's posting address and with the original
 |From: address added to the addresses in the original Reply-To:
 |header and with Content-Type: 

Re: [nmh-workers] FSF is changing Mailman list settings unless you opt out (fwd)

[Steffen Nurpmeso]

Paul Vixie wrote in <2b52ccf2-a559-9b70-869c-68da37fee...@redbarn.org>:
 |Ken Hornstein wrote on 2019-09-26 09:36:
 |> Everyone,
 |> 
 |> I received this email, and I wanted to pass it along.  The executive
 |> summary is: in the near future subject lines to nmh-workers will no
 |> longer be prefixed with "[nmh-workers]" and there won't be a footer
 |> at the end of the message anyone saying that this is the nmh-workers
 |> mailing list.  You can read the details in the message for the complete
 |> technical reasons why this is happening; the other option is to do
 |> what is called "Munge From" and I personally think this is 100x worse
 |> (I am on a Yahoo mailing list where this is done and I hate it).  If
 |> people think we should switch to "Munge From" for this mailing list,
 |> then please make your case here.
 |> 
 |> --Ken
 |
 |yahoo is way off the reservation with this. they aren't growing fast 
 |enough for me to care which of their users can't join mailing lists i 
 |operate -- so my first mover advantage dominates the outcome.
 |
 |anyone who wants me to change a protocol or a norm so that more of the 
 |internet can be centralized by monetizers can go pound sand. (same for 
 |EDNS client subnet.)

YEAH.  Guess that was what i was trying to say.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

-- 
nmh-workers
https://lists.nongnu.org/mailman/listinfo/nmh-workers

Re: format and output all received: lines in an e-mail message

[Steffen Nurpmeso]

Andy Bradford wrote in <20191122084003.18170.qmail@angmar.bradfordfamily\
.org>:
 |Thus said Steffen Nurpmeso on Thu, 21 Nov 2019 23:14:29 +0100:
 |> I have heared someone revived qmail  and wants to include some patches
 |> for builtin TLS etc. That sounded very much interesting, especially if
 |> its mailing-list manager would be maintained again!
 |
 |There has been some momentum around  creating a newer version that has a
 |forward path. I imagine there are  still some diehards out there like me
 |who  just continue  to  use  the original  software  (mine is  minimally
 |patched for example) because it "just works."

It was notqmail..  Not touched in a few months.  Well.

 |I  still use  unpatched ezmlm  for  my own  personal uses  for the  same
 |reasons,  though I  have added  an  extra binary;  but the  rest of  the
 |additional things that were added to it I haven't needed.

I was thinking about that when i practically had to move away from
Sourceforge, since Unicode.org used it for many years, and i had
no rememberance of any issue (except seeing never seen From_ quoting
via space not >).  (Not that i remember any issues after their
switch to Mailman.)  I did not dare to go that route by then.

 |Curiously, the email  you sent me appears to have  never arrived via the
 |MLM that nmh-workers uses---I suspect it has one of those fancy features
 |that thinks an email that was sent to one address shouldn't be also sent
 |to another address (i.e. if the address of the To/Cc recipient is also a
 |member  of  the MLM  exclude  one  of  them).  Personally, I  prefer  to
 |distinguish between direct  replies and replies via MLM so  I don't mind
 |the extra because my filters funnel things appropriately.

That must be your very personal Mailman config then, or?
Well, hmmm ..
I prefer people using Mail-Followup-To: instead of some ML
software modifying the address lists, they could as well just
avoid resending the mail!?!  Yes, i mean, well.
I really like looking into old archives and i hope what i see is
the real original thing, which i think is a value by itself.
But the world does not seem to cherish this.
A nice weekend!

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: format and output all received: lines in an e-mail message

[Steffen Nurpmeso]

Andy Bradford wrote in <20191121072709.1303.qm...@angmar.bradfordfamily.org>:
 |Thus said Greg Minshall on Wed, 20 Nov 2019 10:41:34 +0530:
 |> then, i'd like  to use something like fmttest(1) to  print out all the
 |> "Received:" lines in an e-mail message. ideally, each "Received:" line
 |> would come  out on a  separate line; less  ideally, but i'm  sure very
 |> practical, a very  long line would come out, with  some odd ascii code
 |> separating the individual lines.
 |
 |I usually use  822field (from mess822 [1]) for this  kind of thing which
 |takes all received lines and reformats them one per line (odd ascii code
 |separating them is a newline).
 |
 |For example, your message looks like:
 |
 |$ 822field received < `mhpath cur` | tail -6   
 | from eggs.gnu.org ([2001:470:142:3::10]:33280) by lists.gnu.org with \
 ...

Well, yes, it is a bit restricted, but you could also use S-nail
v14.9.11 or later:

  $ printf 'dig c 25 -;dig 25 h s received;digmsg r 25' |\
s-nail -#Rf +download
  212 Received
  from lists.gnu.org (lists.gnu.org [209.51.188.17]) by sdaoden.eu (Postfix) 
with ESMTPS id 977431604A for ; Thu, 21 Nov 2019 15:28:01 
+0100 (CET)
  from localhost ([::1]:41250 helo=lists1p.gnu.org) by lists.gnu.org with esmtp 
(Exim 4.90_1) (envelope-from 
) id 1iXnR9-000318-IA for 
stef...@sdaoden.eu; Thu, 21 Nov 2019 09:27:59 -0500
  from eggs.gnu.org ([2001:470:142:3::10]:39911) by lists.gnu.org with esmtp 
(Exim 4.90_1) (envelope-from ) id 1iXnQr-0002zm-Am for 
nmh-workers@nongnu.org; Thu, 21 Nov 2019 09:27:42 -0500
  from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from 
) id 1iXnQq-0007yp-1j for nmh-workers@nongnu.org; Thu, 
21 Nov 2019 09:27:41 -0500
  from pellegrino.bradfordfamily.org ([208.53.44.231]:50897) by eggs.gnu.org 
with smtp (Exim 4.71) (envelope-from ) id 
1iXnQp-0007uL-NM for nmh-workers@nongnu.org; Thu, 21 Nov 2019 09:27:39 -0500
  (qmail 73392 invoked from network); 21 Nov 2019 14:27:26 -
  from localhost (HELO arnor.bradfordfamily.org) (127.0.0.1) by localhost with 
SMTP; 21 Nov 2019 14:27:26 -
  (qmail 71469 invoked from network); 21 Nov 2019 14:27:10 -
  from localhost (HELO edoras.bradfordfamily.org) (127.0.0.1) by localhost with 
SMTP; 21 Nov 2019 14:27:10 -
  (qmail 50445 invoked from network); 21 Nov 2019 14:27:10 -
  from angmar.bradfordfamily.org (50.77.44.21) by edoras.bradfordfamily.org 
with SMTP; 21 Nov 2019 14:27:10 -
  (qmail 27684 invoked by uid 1000); 21 Nov 2019 14:27:10 -

Where dig is actually digmsg, c is create, h is header, s is show,
and r is remove.  String matching etc. via the csop and vexpr
commands.
I have heared someone revived qmail and wants to include some
patches for builtin TLS etc.  That sounded very much interesting,
especially if its mailing-list manager would be maintained again!

 |Andy
 |
 |[1] https://cr.yp.to/mess822.html
 ...

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: format and output all received: lines in an e-mail message

[Steffen Nurpmeso]

Valdis Klētnieks wrote in <235182.1574454918@turing-police>:
 |On Fri, 22 Nov 2019 20:51:52 +0100, Steffen Nurpmeso said:
 |
 |> I prefer people using Mail-Followup-To: instead of some ML
 |> software modifying the address lists, they could as well just
 |> avoid resending the mail!?!  Yes, i mean, well.
 |> I really like looking into old archives and i hope what i see is
 |> the real original thing, which i think is a value by itself.
 |> But the world does not seem to cherish this.
 |
 |A bigger issue is GMail's handling of Message-Id:
 |
 |If you post to a mailing list, it notes the Message-Id: on the way out,
 |and when the list sends its copy to you, it gets silently munched by the
 |duplicate suppressor.  So unless you are careful with Fcc: and have
 |nmh save a copy in the appropriate folder, your comments in a thread
 |go poof.

Poof also in use to spring out of existence.
To me such situations happen regulary even without GMail ._.

Before the upcoming release of my MUA i will have to add
a reply-to-swapin variable so one can use the address in Reply-To:
not only as an exclusive receiver override (as with M-F-T:), but
only to replace the one address in From: (or Sender:) with it.
This is the only idea i have to work with DKIM++ the way it is in
use on those lists which use the "x via y" syntax.
(Even if noone wants to hear just about any critics regarding
DKIM / DMARC / ARC, neither here nor on other lists.)

 --End of <235182.1574454918@turing-police>

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: nmh imap gpg

[Steffen Nurpmeso]

john doe wrote in <9019622b-d084-25ad-f574-c5a69be76...@mail.com>:
 |On 1/8/2020 4:19 PM, Steffen Nurpmeso wrote:
 |> john doe wrote in :
 |>|Hello all, and thank you for your answers.
 |>|
 |>|On 1/8/2020 3:07 PM, Ralph Corderoy wrote:
 |>|>> As far as I understand it, NMH can not be used directly with IMAP,so I
 |>|>> would like to use FDM for this.
 |>|>
 |>|> For those of us that didn't know, or have forgotten, FDM is a
 |>|> fetchmail(1)-alike.  https://github.com/nicm/fdm#readme
 |>|
 |>|Is there any alternative to fdm for imap that works well with NMH?
 |>
 |> What do you mean by that?
 |
 |I'm planning to use nmh, which does not support the imap protocol.
 |
 |One way to use imap with nmh is to use fdm but other then that, I was
 |wondering what else could be used with nmh to access my e-mail using imap?

That much i had understood.  The question that crossed my mind was
just what features etc. you need.  For example, we cannot simply
pass a mail to some consumer, and we do also not support the nmh
storage format, so either of an intermediate MBOX or Maildir
storage is required.  But of course you can do things like

  s-nail -A my_account -Y 'file IMAP-ACCOUNT' \
-Y 'copy :n %' -Y quit || exit 1

to get new mail copied over to your local $MAIL, and the ||exit
clause will become increasingly reliable.  Anything better has to
wait until after v15, which hopefully comes, somewhen.

 |Thanks for s-nail, 'mailx' could do what I want imap/smtp/gpg.

Tja, thanks for the thanks, that mailx renaming will take some
time.  And gpg will not happen before, i hope this summer.  And of
course that has nothing to do with nmh integration then, you would
need to compose and view messages from within the (then still
pretty much restricted) MUA.

Maybe funnily, just this day i have seen the first time a message
generated by Cyrus-JMAP/3.1.7 (on a FreeBSD list).  I would expect
the JMAP protocol to join mail and calendar and contacts under an
umbrella protocol series, and thus turn IMAP into a vintage thing,
maybe even fast.  (And even though a new IMAP revision is to be
expected this year, i think.)

 |John Doe
 --End of <9019622b-d084-25ad-f574-c5a69be76...@mail.com>

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: nmh imap gpg

[Steffen Nurpmeso]

john doe wrote in :
 |Hello all, and thank you for your answers.
 |
 |On 1/8/2020 3:07 PM, Ralph Corderoy wrote:
 |> Hi John,
 |>
 |>> As far as I understand it, NMH can not be used directly with IMAP,so I
 |>> would like to use FDM for this.
 |>
 |> For those of us that didn't know, or have forgotten, FDM is a
 |> fetchmail(1)-alike.  https://github.com/nicm/fdm#readme
 |
 |Is there any alternative to fdm for imap that works well with NMH?

What do you mean by that?  My MUA inherited IMAP support from its
predecessor Heirloom mailx (now S-nail for some more time), it is
primitive but usable.  Error handling and recovery of that entire
"module" is primitive / non-existing, i.e., for superior
scriptable access.  But beside that it is a basic yet functional
IMAP implementation.

(It actually has even some neat ideas, like a local IMAP cache.
But these will vanish in the future.)

 |>> John Doe
 |>
 |
 |Are you refering to the signature?
 |
 |> You may be interested to know that the mailing list got a few, not many,
 |> bounces from subscribers because they thought your email was spam;
 |> I suspect it was the moniker that triggered them, though things like the
 |> all-CAPS subject raised the score here.
 |>
 |
 |At the time I created this e-mail address, I thought that using "john
 |doe" was a good idea, now that I'm using this e-mail more and more, it
 |became apparent that I neglected spam filtering! :)
 |
 |Sorry about that.
 |
 |P.S.
 |
 |Duly noted for not capitalizing subject anymore.
 |
 --End of 

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhlogin stopped working

[Steffen Nurpmeso]

Ken Hornstein wrote in
<20200720191854.ea8a179...@pb-smtp1.pobox.com>:
 |>I noticed that my two gmail accounts stopped working.  I can mhlogin
 |>again, and was told:
 |>
 |>   Sign in with Google temporarily disabled for this app This app has
 |>   not been verified yet by Google in order to use Google Sign In.
 |
 |Sigh.  I guess the shoe has finally dropped.
 |
 |We need to do "something" to verify nmh as a valid Google app.  I wasn't
 |in charge of that, and my understanding was the last time we tried to do
 |that it got rejected for some strange reason.
 |
 |Anybody know what we need to do here to fix this?

In the documentation of my MUA i link to

  https://github.com/google/gmail-oauth2-tools/wiki/OAuth2DotPyRunThrough

For mutt(1) someone put quite some effort to generate a full
OAuth2 support script if i understood right what "flew by" in my
inbox.  It should be in contrib, there.
Well, and in the manual of the MUA i maintain you could read

 But, how about XOAUTH2 / OAUTHBEARER?
   Following up I cannot login to Google mail (via OAuth)[43] one OAuth-
   based authentication method is available: the OAuth 2.0 bearer token
   usage as standardized in RFC 6750 (according SASL mechanism in RFC 7628),
   also known as XOAUTH2 and OAUTHBEARER, allows fetching a temporary access
   token via the web that can locally be used as a password[497].  The pro‐
   tocol is simple and extendable, token updates or even password changes
   via a simple TLS secured server login would be possible in theory, but
   today a web browser and an external support tool are prerequisites for
   using this authentication method.  The token times out and must be peri‐
   odically refreshed via the web.

   Some hurdles must be taken before being able to use this method.  Using
   GMail as an example, an application (that is a name) must be registered,
   for which credentials, a “client ID” and a “client secret”, need to be
   created and saved locally (in a secure way).  These initial configuration
   steps can be performed at
 https://console.developers.google.com/apis/credentials.
   Thereafter a refresh token can be requested; a python program to do this
   for GMail accounts is
 https://github.com/google/gmail-oauth2-tools/raw/master/python/
 oauth2.py:

 $ python oauth2.py --user=EMAIL \
   --client-id=THE-ID --client-secret=THE-SECRET \
   --generate_oauth2_token
 To authorize token, visit this url and follow the directions:
   https://accounts.google.com/o/oauth2/auth?client_id=...
   Enter verification code: ...
   Refresh Token: ...
   Access Token: ...
   Access Token Expiration Seconds: 3600
 $ # Of which the last three are actual token responses.
 $ # Thereafter access tokens can regulary be refreshed
 $ # via the created refresh token (read on)

   The generated refresh token must also be saved locally (securely).  The
   procedure as a whole can be read at
 https://github.com/google/gmail-oauth2-tools/wiki/
 OAuth2DotPyRunThrough.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhlogin stopped working

[Steffen Nurpmeso]

Ken Hornstein wrote in
<20200720194227.63d9363...@pb-smtp2.pobox.com>:
 |>For mutt(1) someone put quite some effort to generate a full
 |>OAuth2 support script if i understood right what "flew by" in my
 |>inbox.  It should be in contrib, there.
 |
 |We've DONE that work.

Would you mind not always screaming at me, thank you.
It is not only an updater, it is a fat script which also handles
Microsoft, you know.

 |>   Some hurdles must be taken before being able to use this method.  Using
 |>   GMail as an example, an application (that is a name) must be registered\
 |>   ,
 |
 |This step is the problem.  We were registered but now we aren't.  I wasn't
 |part of that process; hey guys, can we try to address this?

So what have you DONE then?  Maybe he needs to do it for his own
account, they always switch my one back, just today they did.
(Regarding lesser secure apps for my PLAIN only login account.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhlogin stopped working

[Steffen Nurpmeso]

Michael Richardson wrote in
<5853.1595289693@localhost>:
 |
 |> Some hurdles must be taken before being able to use this method.  Using
 |> GMail as an example, an application (that is a name) must be registered,
 |> for which credentials, a “client ID” and a “client secret”, need to be
 |> created and saved locally (in a secure way).  These initial configuration
 |> steps can be performed at
 |
 |But we can't embed a client secret in open source.

Well i did it "for me" (registering the name) only.
It was hard enough to find that developer page and "do it" anyway.

 |Google should know better than this.
 |I'm back to fetchmail with POP3 and the application password.

Google has some trouble, at least for my account.
I still use PLAIN if it is advertised, because i for one do not
like OAuth authentication.  (The Alpine MUA for example included
code to do HTTP queries and JSON interpretation just to be able to
deal with XOAUTH/OAUTHBEARER responses, which are just key/value
pairs.)  And this is what happens since yesterday (once
i encountered it, i do not use that account very often):

  s-nail: >>> SERVER: 220 smtp.gmail.com ESMTP x4sm15187242eju.2 - gsmtp
  s-nail: *smtp-config*: using ehlo extension
  s-nail: >>> EHLO gmail.com
  s-nail: >>> SERVER: 250-smtp.gmail.com at your service, [217.144.132.164]
  s-nail: >>> SERVER: 250-SIZE 35882577
  s-nail: >>> SERVER: 250-8BITMIME
  s-nail: >>> SERVER: 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN 
OAUTHBEARER XOAUTH
  s-nail: SMTP: authentication: skipping PLAIN-CLIENTTOKEN
  s-nail: SMTP: authentication: skipping XOAUTH
  s-nail: >>> SERVER: 250-ENHANCEDSTATUSCODES
  s-nail: >>> SERVER: 250-PIPELINING
  s-nail: >>> SERVER: 250-CHUNKING
  s-nail: >>> SERVER: 250 SMTPUTF8
  s-nail: *smtp-config*: using pipelining extension
  s-nail: *smtp-config*: using auth extension
  s-nail: SMTP: authentication: selecting PLAIN
  s-nail: >>> AUTH PLAIN 
  s-nail: >>> MAIL FROM:
  s-nail: >>> RCPT TO:
  s-nail: >>> DATA
  s-nail: >>> SERVER: 451 4.5.0 SMTP protocol violation, see RFC 2821 
x4sm15187242eju.2 - gsmtp
  s-nail: SMTP: unexpected status from server: 451 4.5.0 SMTP protocol 
violation, see RFC 2821 x4sm15187242eju.2 - gsmtp

No no.  Or

  s-nail: TLS connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
  s-nail: >>> SERVER: 220 smtp.gmail.com ESMTP q24sm17507947edg.51 - gsmtp
  s-nail: *smtp-config*: using ehlo extension
  s-nail: >>> EHLO gmail.com
  s-nail: >>> SERVER: 250-smtp.gmail.com at your service, [217.144.132.164]
  s-nail: >>> SERVER: 250-SIZE 35882577
  s-nail: >>> SERVER: 250-8BITMIME
  s-nail: >>> SERVER: 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN 
OAUTHBEARER XOAUTH
  s-nail: SMTP: authentication: skipping PLAIN-CLIENTTOKEN
  s-nail: SMTP: authentication: skipping XOAUTH
  s-nail: >>> SERVER: 250-ENHANCEDSTATUSCODES
  s-nail: >>> SERVER: 250-PIPELINING
  s-nail: >>> SERVER: 250-CHUNKING
  s-nail: >>> SERVER: 250 SMTPUTF8
  s-nail: *smtp-config*: using pipelining extension
  s-nail: *smtp-config*: using auth extension
  s-nail: SMTP: authentication: selecting PLAIN
  s-nail: >>> AUTH PLAIN ..
  s-nail: >>> MAIL FROM:
  s-nail: >>> RCPT TO:
  s-nail: >>> DATA
  s-nail: >>> SERVER: 534-5.7.14 
>> SERVER: 220 smtp.gmail.com ESMTP q25sm15582101edb.58 - gsmtp
  s-nail: *smtp-config*: using ehlo extension
  s-nail: >>> EHLO gmail.com
  s-nail: >>> SERVER: 250-smtp.gmail.com at your service, [217.144.132.164]
  s-nail: >>> SERVER: 250-SIZE 35882577
  s-nail: >>> SERVER: 250-8BITMIME
  s-nail: >>> SERVER: 250-AUTH LOGIN PLAIN XOAUTH2 PLAIN-CLIENTTOKEN 
OAUTHBEARER XOAUTH
  s-nail: SMTP: authentication: skipping PLAIN-CLIENTTOKEN
  s-nail: SMTP: authentication: skipping XOAUTH
  s-nail: >>> SERVER: 250-ENHANCEDSTATUSCODES
  s-nail: >>> SERVER: 250-PIPELINING
  s-nail: >>> SERVER: 250-CHUNKING
  s-nail: >>> SERVER: 250 SMTPUTF8
  s-nail: *smtp-config*: using auth extension
  s-nail: SMTP: authentication: selecting PLAIN
  s-nail: >>> AUTH PLAIN .
  s-nail: >>> SERVER: 534-5.7.14 

Re: Bug reported regarding Unicode handling in email address

[Steffen Nurpmeso]

Steffen Nurpmeso wrote in
 <20210614162626.vfjxt%stef...@sdaoden.eu>:
 ...
 | <20210614121214.84c1621...@orac.inputplus.co.uk>:
 ...
 ||Why not iconv(3) the input from the user's locale, the MIME part's
 ||charset, etc., to UTF-8, work internally, and then iconv() again on the
 ...
 |functions do not support graphemes, and __STDC_ISO_10646__ is an
 |option, so that you cannot simply code some tables on your own to
 |fill the gaps, because looking at the wchar_t codepoints may not
 |give you a Unicode "codepoint" (though maybe all do it like that
 |so in practice you could make this a precondition).  I had to

To add that if i recall correctly citrus for example does this,
using the upper bits of wchar_t for state info, but i have
forgotten whether that was done in an UTF-8 locale, or rather in
CJK or SHIFT-JS or whatever (my gut says the latter).

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Bug reported regarding Unicode handling in email address

[Steffen Nurpmeso]

Good evening.

Ken Hornstein wrote in
 <20210615025348.83822125...@pb-smtp20.pobox.com>:
 |>kre was coming from a "per draft source character set" i think.
 |>But of course, application dependent.  It is more general than "i
 |>really need this now to get nmh (or mailx) going".  When i went
 |>online around 2010 there was a Python member (Murray, who did the
 |>rewrite of the Python mail engine) who was (or is) an nmh user, as
 |>he said.  I looked at nmh but i think it could not even do MIME by then?
 |
 |It depends on what you mean.  nmh, back when it was MH, could do MIME,
 |but the support was ... not wonderful, if you wanted to deal with modern
 |messages.  It is better now.
 |
 |>But even with only columns there are problems, like bidi.
 |
 |In one sense, we kinda don't have to deal with this because we just feed
 |our output into a pager.  Probably in theory we could do better, but we
 |have a ways to go until our MIME support is good enough to deal.
 |
 |>For serialization you are surely right.  This imposes a conversion
 |>back and forth to wchar_t with POSIX interface, then.  And you
 |>have already lost the performance battle.
 |
 |Are you running on super slow machines?  I can't really imagine that
 |really impacting performance in any measurable way.

We turn in circles, and obviously off-topic for the nmh list.

 |>So UTF-8 pretties up stuff for an all-american or all-english view.
 |
 |Well, guilty on both counts, but I think if you're concerned about size
 |then it probably is more compact for most European languages as well.

Well it is cool, it would be nice if it would have been adopted
more widely shortly after it was invented.  C90 Amendment 1 would
have been great, as Plan9 shows.  Despite that the number is
a minority by very far, and in the 60s an african bishop said "by
the year 2700 the white man will have destroyed live on earth",
and i believed him already when i was young.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Bug reported regarding Unicode handling in email address

[Steffen Nurpmeso]

Steffen Nurpmeso wrote in
 <20210615212220.5bc88%stef...@sdaoden.eu>:
 ...
 |a minority by very far, and in the 60s an african bishop said "by
 |the year 2700 the white man will have destroyed live on earth",
 |and i believed him already when i was young.

Actually he said

  By the year 2700 the white man will have destroyed live on
  earth, and then the time of the Africans begins.

To clarify that.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Bug reported regarding Unicode handling in email address

[Steffen Nurpmeso]

Ralph Corderoy wrote in
 <20210614205202.034db21...@orac.inputplus.co.uk>:
 |Hi Steffen,
 |
 |> It is still hard to do with POSIX let alone ISO.  You need an UTF-8
 |> locale you can actively select, POSIX/ISO functions do not support
 |> graphemes, and __STDC_ISO_10646__ is an option, so that you cannot
 |> simply code some tables on your own to fill the gaps, because looking
 |> at the wchar_t codepoints may not give you a Unicode "codepoint"
 |> (though maybe all do it like that so in practice you could make this a
 |> precondition).
 |
 |Thanks for the detail, I see you point now.

Oh, i was almost a hundred percent sure you saw it before already ;)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Bug reported regarding Unicode handling in email address

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20210614165452.a056f120...@pb-smtp20.pobox.com>:
 |>Sure, convert to Unicode, work in Unicode, convert back, that is
 |>the way to go.
 |
 |I know that this is application dependent, but what "work" do you
 |need to perform on the characters?
 |
 |I realized back when I was originally looking at i18n issues in nmh we
 |don't need to perform THAT much work on characters internally.  We DO
 |do some work when it comes to calculating character width in the format
 |engine, but that's all in the native character set.  So I realized that
 |at least for nmh, there's no advantage to converting to Unicode/UTF-8
 |internally, and a number of disadvantages; like you say, the xlocale
 |functions are non-portable and you can't really get there with the
 |existing POSIX APIs.  Converting internally to Unicode would force you
 |to depend on something like ICU.

kre was coming from a "per draft source character set" i think.
But of course, application dependent.  It is more general than "i
really need this now to get nmh (or mailx) going".  When i went
online around 2010 there was a Python member (Murray, who did the
rewrite of the Python mail engine) who was (or is) an nmh user, as
he said.  I looked at nmh but i think it could not even do MIME by then?
Granted i do not know much of nmh.  You definetely need is-space
for line break detection, and if you visualize yourself you need
is-print or is-control etc.  Etc etc, you know at least as well as
i do.  'Just saying.

But even with only columns there are problems, like bidi.
I added "headline-bidi" "support"

  In general setting this variable will cause Mailx to encapsulate
  text fields that may occur when displaying headline[435] (and some
  other fields, like dynamic expansions in prompt[517]) with special
  Unicode control sequences; it is possible to fine-tune the terminal
  support level by assigning a value: no value (or any value other
  than ‘1’, ‘2’ and ‘3’) will make Mailx assume that the terminal is
  capable to properly deal with Unicode version 6.3, in which case
  text is embedded in a pair of U+2068 (FIRST STRONG ISOLATE) and
  U+2069 (POP DIRECTIONAL ISOLATE) characters.  In addition no space
  on the line is reserved for these characters.

  Weaker support is chosen by using the value ‘1’ (Unicode 6.3, but
  reserve the room of two spaces for writing the control sequences
  onto the line).  The values ‘2’ and ‘3’ select Unicode 1.1 support
  (U+200E, LEFT-TO-RIGHT MARK); the latter again reserves room for
  two spaces in addition.

but it is no good here (st(1)).  Best is without it :)

  From steffen Tue Jun  3 13:42:08 2014
  Date: Tue, 03 Jun 2014 13:42:08 +0200
  From: =?utf-8?B?2KPYrdmF2K8g2KfZhNmF2K3ZhdmI2K/Zig==?= 
  To: =?utf-8?B?2KPYrdmF2K8g2KfZhNmF2K3ZhdmI2K/Zig==?=
  Subject: =?utf-8?B?2KPYrdmF2K8g2KfZhNmF2K3ZhdmI2K/Zig==?=
  MIME-Version: 1.0
  Content-Type: multipart/mixed;
   boundary="=_01401795729=-WIIWUCvp3AwFMhX+fbN+aN6QsACHfW=_"
  Status: R

  This is a multi-part message in MIME format.

  --=_01401795729=-WIIWUCvp3AwFMhX+fbN+aN6QsACHfW=_
  Content-Type: text/plain; charset=UTF-8
  Content-Transfer-Encoding: 8bit
  Content-Disposition: inline

  أحمد المحمودي.

  --=_01401795729=-WIIWUCvp3AwFMhX+fbN+aN6QsACHfW=_
  Content-Type: text/plain; charset=UTF-8
  Content-Transfer-Encoding: 8bit
  Content-Disposition: attachment;
   filename="أحمد المحمودي.txt"

  أحمد المحمودي.

  --=_01401795729=-WIIWUCvp3AwFMhX+fbN+aN6QsACHfW=_--

Nah, *really* proper internationalization is a very complicated
thing, but it seems you can get away with only slightly touching
this in an email program unless you display or edit actual text.
As i have no right-to-left capabilities, i cannot test anyway.

 |>Really, the older i get the more i think that UTF-16 is not the
 |>worst decision regarding Unicode.  Surrogate pairs have to be
 |>handled, but for UTF-8 you always have to live with multibyte
 |>anyway.
 |
 |I guess I think out of all of the possible worlds, UTF-8 is probably
 |the best compromise.

For serialization you are surely right.  This imposes a conversion
back and forth to wchar_t with POSIX interface, then.  And you
have already lost the performance battle.

You know, that is _really_ weird.  Remembering NetBSD pimping
their vis(3) (i was subscribed to their source-changes for years),
vis(3) goes through this:

/* Allocate space for the wide char strings */
psrc = pdst = extra = NULL;
mdst = NULL;
if ((psrc = calloc(mbslength + 1, sizeof(*psrc))) == NULL)
return -1;
if ((pdst = calloc((16 * mbslength) + 1, sizeof(*pdst))) == NULL)
goto out;
if (*mbdstp == NULL) {
if ((mdst = calloc((16 * mbslength) + 1, sizeof(*mdst))) == NULL)
goto out;
*mbdstp = mdst;
}

And you do not want to look at the rest.  I mean, wow!, that
entirely hammers you off the map!  And not to talk about
longjmp(3) or other signal mess.

The good thing 

Re: Sharding by Year.

[Steffen Nurpmeso]

Michael Richardson wrote in
 <31957.1623081930@localhost>:
 |Ralph Corderoy  wrote:
 |>> outgoing/y2001  has 1287 messages  (1-1287).
 |>> outgoing/y2002  has 9335 messages  (1-9335).
 |>> outgoing/y2003  has 3909 messages  (1-3909).
 |
 |> Out of interest, how did you pick what's in each year?  Textually, with
 |> ‘2002’ appearing in the Date field.  Or as a year and if so was that
 |> local time, the sender's time, or UTC?  :-)
 |
 |Well, it's outgoing email, so it's all mine, so it's all in my time zone.
 |
 |On Jan. 1, I do:
 |  refile -src +outgoing all +outgoing/y2020

Damn, i just call a .rotate.sh asap, this year at 00:11:19.
Relevant part is

  cd mail || exit 4

  find . -maxdepth 1 -type f \
-not -name 'download' -not -name '*gmane*' \
-not -name '*.xz' -not -name '*.zst' \
-not -name '*.sh' -not -name .yearstamp |
  while read f; do
f=`basename ${f}`
if echo ${f} | grep -E '^[1-9][0-9]{3}\.'; then
  :
elif [ -s ${f} ]; then
  echo "Rotating ${f} to ${YEAR}.${f}.zst"
  zstd -T0 -19 -zc < ${f} > ${YEAR}.${f}.zst
  chmod 0400 ${YEAR}.${f}.zst
  : > ${f}
else
  echo "${f} is an empty file, removing it"
  rm ${f}
fi
  done

  echo $YEAR > .yearstamp

Of course sometimes threads cross the line.  Thus..

 |well, I often forget Jan. so, "all" might be adjusted having run "scan"
 |
 |It would be nice to be able to treat these folders a single folder for
 |certain operations.

Oh i wish my MUA could do this yet.  For many years in TODO:

 The absolute sensation would be joinable operations over multiple
 open mailboxes, e.g., views over multiple such!

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Bug reported regarding Unicode handling in email address

[Steffen Nurpmeso]

Ralph Corderoy wrote in
 <20210612103715.a572c21...@orac.inputplus.co.uk>:
 |>> I am aware that some people, for reasons I cannot comprehend, want
 |>> to run in the "C" locale
 |>
 |> I do that, not so much because I want to, but because that's what
 |> happens when no LC_* env variables (nor LANG) exist at all.   That's
 |> me.   I believe you understand that locales aren't exactly first class
 |> objects in NetBSD...  (Or not yet anyway).
 |
 |https://wiki.netbsd.org/tutorials/unicode/ suggests Unicode through
 |UTF-8 is well supported as long as the user sets the appropriate
 |environment variables.  Isn't just that you choose not to set them?

Last i looked they use a gigantic chunk of memory in mbstate_t or
so (128 byte?).  Other than that the Citrus project was ..the
first to support locales in (free) Unix?  I think so.  What was
totally missing was support for collation.  Understandable here
especially strxfrm(3) which uses a terrible algorithm that drives
me up the wall in order to turn some A in a B that can be matched
via strcmp(3).  /ME shivers.  Other than that the w*() interface
is a terrible mess, it does not know about graphemes,
normalization, de-/composing, etc.  Just my one cent.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

RFC 9057 (the Author: header field)

[Steffen Nurpmeso]

Hello.

I seem to recall that you are slowly drifting towards a new
release, and as i was thrilled by RFC 9057 i thought i bring it to
your attention, in case you are not yet aware.  I implemented it
for my mailer right away yesterday afternoon (i have not restarted
it yet though), as i think it good stuff that brings the email
protocol back into good shape, after it has been messed for DMARC
or whatever, resulting in all this terribe "x via y" From: lines,
and the misuse of Reply-To: etc etc.
(My support is not yet complete, replying to emails uses an
interpretation of the standard that noone else uses, and the
change is incomplete in the queue since last September or so.  It
now has to / will take Author: into account right away.)

Ciao,

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: RFC 9057 (the Author: header field)

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20210624180220.daccf12a...@pb-smtp20.pobox.com>:
 |>I seem to recall that you are slowly drifting towards a new
 |>release, and as i was thrilled by RFC 9057 i thought i bring it to
 |>your attention, in case you are not yet aware.  I implemented it
 |>for my mailer right away yesterday afternoon (i have not restarted
 |>it yet though), as i think it good stuff that brings the email
 |>protocol back into good shape, after it has been messed for DMARC
 |>or whatever, resulting in all this terribe "x via y" From: lines,
 |>and the misuse of Reply-To: etc etc.
 |
 |I THINK ... we can basically implement this via changing the mh-format
 |templates?

If generated it should be identical to From: (not Sender:).  When
replying my interpretation would be that it should be used in
favour of From: if it exists, and no reply-to: or
mail-followup-to: or however complicated the logic is, .. so
i would like to think you do not need to write C code in nmh?

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhbuild: extraneous information in message

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20210512224215.9969ccd...@pb-smtp1.pobox.com>:
 |>nmh could adapt or build upon the src/mx/mailcap.c of my MUA.  It
 |>is pretty much standing on its own feet, has matured (there were
 |>a few hmm nits), and offers really nice extensions like
 |>x-mailx-test-once, x-mailx-ignore etc.
 |
 |Thank you for that offer.  And forgive me if this is a stupid question,
 |but ... which MUA is that? :-)

Oh, no problem ;)  This is a mailx clone yet called s-nail.  The
code in question is at [1].  If you are interested in mailcap,
what really may be of interest, is reading our manual, for example
at [2] (there with links), i tried to collect what could be found
"all" around (the same is true for .netrc which follows in the
next section).

  [1] 
https://git.sdaoden.eu/browse?p=s-nail.git;a=blob;f=src/mx/mailcap.c;hb=refs/heads/next
  [2] https://www.sdaoden.eu/code-nail.html#37

P.S.: the formatting of [2] is a bit unfortunate, it is plain
output of groff's html driver which often places indendation of
list items wrong.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhbuild: extraneous information in message

[Steffen Nurpmeso]

Ralph Corderoy wrote in
 <20210512202431.3d70322...@orac.inputplus.co.uk>:
 |Hi Valdis,
 |
 |> Which raises the question - what is getting into the path so when
 |> Laura adds entries to /etc/mailcap, things start working for her?
 |
 |Ken suggested some distros might alter nmh's release when packaging to
 |use mailcap(5).  Looking at
 |https://sources.debian.org/patches/nmh/1.7.1-4/, which I think Laura is
 |uses, nothing stands out as adding that for Debian.

nmh could adapt or build upon the src/mx/mailcap.c of my MUA.  It
is pretty much standing on its own feet, has matured (there were
a few hmm nits), and offers really nice extensions like
x-mailx-test-once, x-mailx-ignore etc.

Despite its deficiencies (as stated on BOF, and that
"nametemplate" format can only be at front as in %s.pdf not
pdf.%s), it parses pretty well and complains on bugs like so, too
(in the RFC 1524 example for example which in turn comes from
decade old predecessor i have forgotten what that was all about).

Of course it sits on our (still deficient) internal infrastructure
for child processes ("test" field), temporary files, LOFI
alloca(3) replacement etc.  Anyhow, if so, should be taken from
[next] branch because there were some cosmetic changes which make
it look nicer. :)  You know.  For example it treats un/even
reverse solidus at EOLs nice, which is still not true as of today
for shells, for example:

  #?0|kent:steffen$ echo du \
  > hey
  du \\hey
  #?0|kent:steffen$ dash
  #kent:$ echo du \
  > das
  du \das
  #kent:$

(That should maybe be reported.  I think i am subscribed, will do.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Calendaring?

[Steffen Nurpmeso]

doug dougwellington.com wrote in
 :
 |I'm an old Unix system admin command line type, and I love MH/nmh.
 |
 |I let the Mac and PC worlds distract me for a bit, but I'm really tired \
 |of dealing with Calendar, Outlook, and the like, especially when Microsoft \
 |is threating to change the user interface again.  Is there a calendaring \
 |program that uses a similar structure to nmh behind the scenes?  If \
 |one doesn't exist, I might have to make one of my own.

remind?  From Diane Skoll (skoll.ca).
(But note: _really_ "similar":

  REM Thu Nov [Week_4]SCANFROM -7 ADDOMIT MSG %"Thanksgiving Day%"
  REM Fri Nov [Week_4+1]  SCANFROM -7 ADDOMIT MSG %"Thanksgiving (cont.)%"
  OMITDec 24  MSG %"Christmas Eve%"
  OMITDec 25  MSG %"Christmas%" Day)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Calendaring?

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20231102201740.80a852f...@pb-smtp21.pobox.com>:
 |>The all-embedded RFC 7265 JCAL (plus JMAP etc) is surely the
 |>future for all of you.
 |
 |I get the feeling there's some scorn in that statement.  I'm not trying
 |to drag anyone else's choices; it's tough to find the right balance
 |between "keeping up with the times" and "sticking with stuff you know
 |that works just fine".  I can only offer (when asked!) how I came to
 |make the decisions I chose.
 |
 |>Unfortunately no Yubikey/-alike thing yet, so i type my harddisk
 |>decryption key, then my password, then the password of an
 |>additional encfs filesystem, and then have a script which loads
 |>keys into ssh-agent aka provides decrypted versions for easy
 |>consumation (via copy+paste or so).  The browser "container" which
 |>(actually) has passwords is special and also stores in such a one.
 |>(Also decrypted by that script.)
 |
 |I can only say that my personal security is NOT designed to defend against
 |nation-state level attacks :-)  But maybe it should be?

Hm, i .. could imagine you are more familiar with these kind of
things, as a higher ranked former army computer specialist, in
reality.  Ie, one hears this and that, in the end.

The topic itself -- i do not know.  I tend to totally agree with
your statement, but then again, not.  I mean there is nothing
i would have to be ashamed of or that could make me blackmailable.
But i do have SSH keys to other people's computer(s) (networks).
I have S/MIME and PGP (and signify) key.
There is access to a bank account, and i make a living from that
alone, one bank, one account.  At some time, make one of three.

I never had cold security (hard disk encryption) in the past, but
then did it in 2021.  It was easy, and these NVME things have
a sheer unbelievable (really: totally mind blowing) I/O
performance regardless.
I did not even have any disk or folder encryption at all until
2019, only the distributed backups were PGP encrypted.
By then i was of the opinion that this is not enough, with the
laptop all over the place, but i do take care quite a bit when
i have the laptop around, so a forced "armed" robbery would have
been necessary.
So first came the encfs, at the same time i replaced backups with
filesystem snapshots.  But the cold security problem was then
nagging, and so i did that, too, keeping the rest "as-is".  Works
just fine.  Surely no protection against forced power against me
as a human factor still.

Btw i kept an email of Theodore Ts'o from openssl-dev@ of 2014,
where he describes his workflow as

  ..
  % git tag -s[.]

  ..
  % git push ssh://[.]

  ..
  (I have aliases and shell scripts for most of this, but I've expanded
  all of this out for clarity.)

Today they all use these cards which even require hands-on.
In this hindsight i am still too relaxed.
I even talked to one Yubikey user who was totally grazy about that
"forced power" issue praying against his wall "i have no idea what
the password is" "i really cannot tell you anything about the
password as i do not know it, i have never seen it" etc.  If they
get him, they get a thing.  That much is plain.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Calendaring?

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20231102175429.174f22e...@pb-smtp21.pobox.com>:
 ...
 |>I'm firmly planted in the Linux, MacOS, Windows, and Android worlds.
 |>So, my current calendar is a piece of paper on a clipboard that I always
 |>carry with me, LOL!  I find it way more expedient to use the clipboard
 |>rather than try to have a to-do list app and a calendar app and remember
 |>which device I'm using at the time and figure out how to keep everything
 |>in sync.
 |
 |I mean, hey, whatever works for you ... but do you not have anyone
 |else who you need to share a calendar with?  To me that is the other
 |key piece.  If it's just you then you can get away with more analog
 |solutions.  FWIW, _if_ you are willing to live in the Apple Universe
 |then the syncing just happens across all of your devices automatically.
 |I just don't think about it; I know that when I (or anyone else in
 |my household) add something to any device then soon-ish it will
 |automatically appear across all devices for everyone in my household.

The all-embedded RFC 7265 JCAL (plus JMAP etc) is surely the
future for all of you.

 |>I really miss my old Windows phone, but I digress.  Does

Even the Fair Phone is only 70% fair i think.
It is a disgrace.

  ...
 |Since you mentioned it ... what do other people use for password
 |management?  I ended up using a solution called "Master Password" which
 |is an open-source password generator which is now called "Spectre"
 |(https://spectre.app) and has a reasonable paid app for mobile devices.
 |But I am wondering what other people do.

Unfortunately no Yubikey/-alike thing yet, so i type my harddisk
decryption key, then my password, then the password of an
additional encfs filesystem, and then have a script which loads
keys into ssh-agent aka provides decrypted versions for easy
consumation (via copy+paste or so).  The browser "container" which
(actually) has passwords is special and also stores in such a one.
(Also decrypted by that script.)
Upon LID close etc all windows are slock(1)ed, all ssh-agents are
cleared, all encfs volumes are unmounted.  So next time i have to
do it all again (less harddisk decryption).
It sounds annoying.  I am looking forward a bit for a cryptocard.
(But NO! keys on smartphone, unless i buy myself a Pinephone;
unfortunately there is no "PinePhone Pro with AlpineLinux" (even
better CRUX), but a "normal" Linux and connectivity only via USB
bus and thus USB-protocol restrained, that i could trust.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhbuild and long header fields

[Steffen Nurpmeso]

Hello.

Philipp wrote in
 <93c190b84a23e620d07b9504f619b1f2.phil...@bureaucracy.de>:
 |[2023-08-24 21:07] Philipp Takacs 
 |> [2023-08-23 14:29] Philipp 
 |>> [2023-08-20 22:14] David Levine 
 |>>> Ken Hornstein wrote:
 ...
 |I have tested this and fixed some bugs. A commit with a test is attached.

I have not looked at your code (for real), but .. i have a test
for the MUA i maintain like

  i=$(awk 'BEGIN{for(i=0; i<92; ++i) printf "0123456789_"}')

and then i use "$i" as a subject.
The problem with RFC 5322 is that artificial spaces may have to be
introduced to break a line without whitespace.  (That is, you have
to break somewhere, and have to start the next line with
a whitespace, but that is included then, modifying the original
data.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: mhbuild and long header fields

[Steffen Nurpmeso]

Steffen Nurpmeso wrote in
 <20230825213130.xthnk%stef...@sdaoden.eu>:
 |Philipp wrote in
 | <93c190b84a23e620d07b9504f619b1f2.phil...@bureaucracy.de>:
 ||[2023-08-24 21:07] Philipp Takacs 
 ||> [2023-08-23 14:29] Philipp 
 ||>> [2023-08-20 22:14] David Levine 
 ||>>> Ken Hornstein wrote:
 | ...
 ||I have tested this and fixed some bugs. A commit with a test is attached.
 |
 |I have not looked at your code (for real), but .. i have a test
 |for the MUA i maintain like
 |
 |  i=$(awk 'BEGIN{for(i=0; i<92; ++i) printf "0123456789_"}')
 |
 |and then i use "$i" as a subject.
 |The problem with RFC 5322 is that artificial spaces may have to be
 |introduced to break a line without whitespace.  (That is, you have
 |to break somewhere, and have to start the next line with
 |a whitespace, but that is included then, modifying the original
 |data.)

That is: you should switch to MIME RFC 2047 then for the purpose
of uniting maybe even.  Or however nmh wants to deal with this.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Export/sync nmh folders to IMAP server

[Steffen Nurpmeso]

Philipp Takacs wrote in
 :
 |[2022-06-29 21:07] Steffen Nurpmeso 
 |> (I personally like a nice and so on, append-only MIME MBOX the
 |> most, and do not understand why people do not like it.
 |
 |Because mbox is a horrible file format. First of all it's only one file
 |so you have to build a folder stucture around it. Next it's hard to

It is a database.

 |delete (or move) mails if they not at the end of the file. These

Well yes you need to rewrite the file.  With Zawinskis Netscape
index approach it could instead be as easy as setting a bit.
Or you could reserve a byte in the Status: field of messages, but
it likely is as non-portable to other MUAs that access the file.

 |problems are not that bad depending on for what you want to use the mbox
 |and can solved by writing more and complex code.

Not at all complex.

 |The main problem is that there is not the mbox format, there are several
 |variants. Some variants don't allow arbatrary mails as content and most
 |(probably all) of them aren't realy specified. It's nearly impossible to
 |determ which variant you have. Therfor most software just implement one
 |variant and handle other variants incorrect. Jamie Zawinski explains the
 |details of the problems quite good[0].

Didn't we have this once in the past already??
To me MBOX is a database, and when you store MIME mail messages
there is no problem, as the format problem does not occur.
Otherwise anyone applies From_ quoting, i even had to readd this
to the MUA i maintain, because people do not want MIME.  So let
them love their ">From" (or even " From" from i think ezml), even
it modifies their thing.  You see this often on majordomo lists
etc.  I do not understand.  Of course when they PGP or S/MIME sign
their message, or use attachments, they do get MIME and then the
problem does not happen.

 |So please stop using mbox.

Not me.

 |Philipp
 |
 |Ps: what bugs me the most is when mbox is used to store exactly one mail

 |Pps: Sorry for the OT rant, but this bugs me every time I get a mbox

I empathise for the one line that makes a difference!

 |[0] https://www.jwz.org/doc/content-length.html
 --End of 

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Export/sync nmh folders to IMAP server

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20220629184548.f030b19f...@pb-smtp20.pobox.com>:
 |>I guess I will try the Maildir hard links script with
 |>cyrus-imapd for providing access to the 6G of mail on my desktop.
 |>I'm okay that it would be read-only... it's probably a feature.
 |
 |I am continually confused why anyone thinks Maildir is a good idea for
 |mailbox storage; I think it is fine for a _maildrop_ (since that was
 |it's intended purpose), but every time I've seen it used as a backend
 |for mailbox storage, you end up cramming a lot of metadata into the
 |individual filenames and nobody seems to quite agree on those metadata
 |semantics.
 |
 |Also, I thought cyrus-imapd uses it's own internal backend for
 |mail storage and Maildir isn't one of them.  Looks like the list of
 |storage types is here:
 |
 |https://www.cyrusimap.org/imap/concepts/deployment/databases.html#storag\
 |e-types
 |
 |You might be thinking of Dovecot which _does_ use Maildir as a backend
 |storage, but my understanding is that Dovecot has the filename metadata
 |problem mentioned above.

Hm, i rewrote Maildir name things of the MUA i maintain, and
i found:

   /* If the seconds part does not work, go deeper.
* We use de-facto standard "maildir - E-mail directory" from the Courier
* mail server, also used by, e.g., Dovecot: sec.MusecPpid.hostname:2,flags.
* However, a different name convention exists which uses
* sec.pid_counter.hostname:2,flags.
* First go for usec/counter, then pid */

I do not know about other name variants.
But i agree that my a_maildir_cmp() for name comparison is a hog.
We also use an array for all the filenames in a box (i want to
implement Zawinski's index for long, but not yet).
(I personally like a nice and so on, append-only MIME MBOX the
most, and do not understand why people do not like it.  Heck, once
i read on a FreeBSD list an administrator saying something like
"... my boss has a 2GB INBOX, it is no fun; we now use
Archaeopterix" or however this dinosaur/software combo was really
named.  What i mean is: if you only append anyway, have a nice
index.  Cheerio.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Surprising MIME Type from Android.

[Steffen Nurpmeso]

Ralph Corderoy wrote in
 <20220725092852.e055e20...@orac.inputplus.co.uk>:
 |> I ran mhstore(1) and was surprised by the ‘81081.2.*’ filename.
 |...
 |>   2 image/*  2985K
 |...
 |> I haven't checked yet, but I assume it violates the RFCs.
 |
 |It does by my reading of
 |https://datatracker.ietf.org/doc/html/rfc2045#section-5.1
 |
 |Something for mhfixmsg(1) to correct?
 ...
 |> Content-Type: image/*; name="20220721_180552.jpg"
 ...

Fwiw, my mailer has a *mime-counter-evidence* variable which tries

  ...
 |Has anyone else got an example?
 |
 |pick -sea 'content-type.*\*'
  ...

It has become better, but i see that setting adjusting MIME types
very often.  (Though most often it is overwriting
application/octet-stream with something more sensible.  Iirc
especially elder Apple Mail's where notorious false claimers.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: nmh 1.8?

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20221228151732.bd63b1d2...@pb-smtp20.pobox.com>:
 ...
 |MUA that generates that header or checks it.  I'm not even sure we
 |calculate the digest correct for text types, it was a mess in terms
 |of implementation, _and_ MD5 is Officially Considered Broken.  Calling
 ...

I would think that finding two plain text files with the same MD5
that a mail message receiver finds an acceptable read is rather
unlikely though.  (Just generally speaking.  CRUX Linux for
example uses signify for package checksums, but still generates
MD5 checksums as a fallback.  CRC32 is also used still, but noone
would claim it is secure.)

Have a good slip/slide.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: (Not-so) hypothetical question: What to do about NULs?

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20230219012125.2e48b1d7...@pb-smtp21.pobox.com>:
 |>Seems to me this is classifcation of attachment data, which will end up
 |>as octet-stream in that case.
 |
 |It's ... a little confusing!
 |
 |>For S-nail we more or less do what Heirloom mailx has done.
 |
 |Well, it seems that in the message lexer if you encounter a NUL you
 |just stop, from a_msg_scan():
 |
 |  cp = mslp->msl_cap->ca_arg.ca_str.s;
 |  if((c = *cp++) != '\0')
 | break;

That seems to come from a command argument parser, not mail
content.  Ah no, no no, wrong code :)
I can assure you that the email

  From reproducible_build Wed Oct  2 01:50:07 1996
  Date: Wed, 02 Oct 1996 01:50:07 +
  From: e...@am.ple
  Subject: s3
  MIME-Version: 1.0
  Content-Type: text/plain; charset=utf-8
  Content-Transfer-Encoding: quoted-printable
  Status: O

  Alo=00ha
  Boom.

is decoded (of course) and displayed with the NUL converted to the
Unicode graphical for NUL.
The same of i make it "binary" and put a real NUL in place of the
=00.

 |It does look like to me that for IMAP and POP a NUL character is handled
 |properly.  But that doesn't answer the question, what do you THINK should

Uh i really had to look and try out whether binary data on the
input side of IMAP or POP3 properly handles embedded NULs.
I would assume yes.  (More or less.)

 |happen?  Should NULs be passed through?  You basically can't use C strings
 |anywhere if you want to handle embedded NULs.

That is true.

 |>The implementation is total crap. (longjmp codebase, data leaks,
 |>blocking I/O, all that (it was).)  All of these (mailbox read,
 |>content-transfer decoding, character set conversion, .. display
 |>preparation) should be "filters" with input and output plugged together,
 |>with internal buffers as necessary.  That is the v15 MIME and I/O layer
 |>rewrite that is not happening for nine years.
 |
 |Sigh, I know the feeling :-/

A nice Sunday is also not a bad thing.
Ciao,

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: (Not-so) hypothetical question: What to do about NULs?

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20230219001921.597ad1e0...@pb-smtp20.pobox.com>:
 ...
 |- mutt
 ...
 |[.]Internally mutt does
 |have an idea if the content contains a NUL (the CONTENT structure contains
 |a 'nulbin' member which contains the number of NUL bytes), but it's not
 |clear to me what happens when a NUL is encountered.

Seems to me this is classifcation of attachment data, which will
end up as octet-stream in that case.

For S-nail we more or less do what Heirloom mailx has done.
For classification purposes we switch to octet-stream.
For display purposes we happily display it after passing it
through some kind of makeprint.

  isuni = ((n_psonce & n_PSO_UNICODE) != 0);
  ...
 if(!iswprint(wc) && wc != '\n' /*&& wc != '\r' && wc != '\b'*/ &&
   wc != '\t'){
if ((wc & ~S(wchar_t,037)) == 0)
   wc = isuni ? 0x2400 | wc : '?';
else if(wc == 0177)
   wc = isuni ? 0x2421 : '?';
else
   wc = isuni ? 0x2426 : '?';
 }else if(isuni){ /* TODO ctext */
/* Need to filter out L-TO-R and R-TO-R marks TODO ctext */
if(wc == 0x200E || wc == 0x200F || (wc >= 0x202A && wc <= 0x202E))
   continue;
/* And some zero-width messes */
if(wc == 0x00AD || (wc >= 0x200B && wc <= 0x200D))
   continue;
/* Oh about the ISO C wide character interfaces, baby! */
if(wc == 0xFEFF)
   continue;
 }

Or, without mb* and wc* sausage,

   {
  int c;
  while(inp < maxp){
 c = *inp++ & 0377;
 if(!su_cs_is_print(c) &&
   c != '\n' && c != '\r' && c != '\b' && c != '\t')
c = '?';
 *outp++ = c;
  }
  out->l = in->l;
   }

This is even a degression against Heirloom mailx that Jörg
Schilling was very dissatisfied about, as the above only handles
ASCII printable regardless of the locale.  (My plan was to write
a CText library for Unicode handling, and it was quite progressed
with only about two months until decomposition and normalization
were implemented (Christmas 2014), when something very bad
happened.  Maybe i will do it someday.  Or simply do what OpenBSD
does and use perl's fantastic Unicode support to generate some
tables.)

The implementation is total crap.  (longjmp codebase, data leaks,
blocking I/O, all that (it was).)  All of these (mailbox read,
content-transfer decoding, character set conversion, .. display
preparation) should be "filters" with input and output plugged
together, with internal buffers as necessary.  That is the v15
MIME and I/O layer rewrite that is not happening for nine years.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: (Not-so) hypothetical question: What to do about NULs?

[Steffen Nurpmeso]

P.S.:

Congratulations to your new release btw.

I have written an OAuth helper in Python3 that suports OAuth for
GMail, Microsoft, Yandex:

  curl -u moon:mars --basic -O 
https://git.sdaoden.eu/browse/s-toolbox.git/plain/oauth-helper.py

It has a "manual" mode where it documents for GMail

  -- How to create a Google registration --

  Go to console.developers.google.com, and create a new project. The name 
doesn't
  matter and could be "mutt registration project".

   - Go to Library, choose Gmail API, and enable it
   - Hit left arrow icon to get back to console.developers.google.com
   - Choose OAuth Consent Screen
  - Choose Internal for an organizational G Suite
  - Choose External if that's your only choice
  - For Application Name, put for example "Mutt"
  - Under scopes, choose Add scope, scroll all the way down, enable the
"https://mail.google.com/; scope
  [Note this only allow "internal" users; you get the same mail usage scope
  by selecting those gmail scopes without any lock symbol!
  Like this application verification is not needed, and "External" can be
  chosen.]
  - Fill out additional fields (application logo, etc) if you feel like 
it
(will make the consent screen look nicer)

Maybe this helps!

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Bakul Shah wrote in
 :
 |On Apr 3, 2023, at 2:40 PM, Ken Hornstein  wrote:
 |> 
 |> tbl (which seems like it has been supported ... forever?) does the hard
 |> work of creating tables for you.  It seems like the right tool for the
 |> job; even I could figure it out.  As Anthony has pointed out mandoc
 |> is the default man page formatter for BSD systems, now MacOS X, and
 |> some Linux distributions.  Seems to me that switching to tbl for those
 |> things is a no-brainer unless I am missing something, which is always
 |> possible!
 |
 |I don't see a tbl command on MacOS (or freebsd, except if you
 |installed groff (or plan9port -- ignore the troff comment!).

mandoc has primitivized versions built-in.

 |Unless you mean replacing the current setup with the output of
 |equivalent tbl syntax and then checking that in? I tried that.
 |mandoc doesn't understand tbl output for the simple test i tried.
 |
 |/usr/local/bin/tbl <

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20230403223814.692aa1f2...@pb-smtp21.pobox.com>:
 |>>> Sorry if I jumped into the middle and missed something, but what about
 |>>> using this to convert once?
 |>>>
 |>>> groff -Thtml
 |>
 |>> I guess my next question is ... what do we do after that?
 |>
 |>I thought if we ran it through with man (nroff/groff) to ascii, then \
 |>we'd get
 |>asciidoc format essentially.  At which point there are tools that \
 |>deal with
 |>further transformations.
 |
 |I ... am not sure that is correct?  The man page examples I see suggest
 |it is closer to something like Markdown.  E.g.:
 |
 |https://docs.asciidoctor.org/asciidoctor/latest/manpage-backend/

Mind you there is this terrible trend in the OSS scene to not even
include manuals no more.  I am *so* delighted that the BTRFS guys
do again for their "progs" (and i could exaggerate a bit that it
possibly has something to do with me asking for this in IRC).
But they all convert to other formats and then leave it to
packagers to convert that to manuals .. or not even that.  Others
do so themselves at least, and provide the balls (git for
example).

This leads to cynical situations.  For example "ipcalc" of RedHat:

  0c09c6c78920290aaa1c3b91c596834080fbf0b0
  2020-06-18 14:50:15 +0200
  ipcalc.1: converted to markdown

  b2003892e7ddaed77a309882bcba06cea31cda0a
  2020-06-24 21:51:54 +0200
  ipcalc.1: removed
...
  322294d7bf61f9fdf0e45e9b6c6013a7c6a35bfd
  2020-10-14 18:33:24 +
  Do not fail to build if ronn is missing
[..the converter that is..]
...
  f14f19f7fbdb83d7ba2cef5de758c649abdce1e8
  2022-11-12 11:07:24 +0100
  updated manual to comply with ronn syntax

HA-HA-HA!!  Somewhat broken for 2 1/2 years.
What a mess.

Good night from Germany.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

George Michaelson wrote in
 :
 |Not to prolong the agony, I tried the example on OSX for man tbl:
 |
 |  .TS
 |  tab(@);
 |  ccc.
 |  This@is@centered
 |  Well,@this@also
 |  .TE
 |
 |It didn't work with the nroff -man they supply. It did work with mandoc

It surely works with "tbl FILE|nroff|more" if there is tbl and
nroff.  Or solely "groff -tTascii FILE|more".

 |So my summary of understanding so far:

Yes i think so.  Note that in

  Hello world.$
  $
  This^Iis^Icentered$
  .sp 0$
  This^Iis^Icentered$
  $
  Well,^Ithis^Ialso$
  $
  Hey.$
  This not$

mandoc deviates from groff unless .ta is used as in 

  Hello world.$
  $
  .ta 4 8$
  This^Iis^Icentered$
  .sp 0$
  This^Iis^Icentered$
  $
  Well,^Ithis^Ialso$
  $
  Hey.$
  This not$

Ie better set tabs explicitly.  Now.  I am not an expert, and my
mandoc is very old, things might have changed.

  $ ll $(command -v mandoc)
  -rwxr-xr-x 1 steffen steffen 1176552 Apr 13  2019 /usr/local/bin/mandoc*

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Robert Elz wrote in
 <8706.1680561...@jacaranda.noi.kre.to>:
 |Date:Mon, 3 Apr 2023 15:10:15 -0700
 |From:Bakul Shah 
 |Message-ID:  
 |
 || mandoc doesn't understand tbl output for the simple test i tried.
 |
 |No, in general mandoc doesn't process *roff input at all.   Its input
 ...
 |ps: Ken, while I am here, forget about translating HTML back into anything
 ...

Its HTML conversion is much superior compared to the groff one.
This is hard to understand actually, i think this is a Brit
Professor who did it, and i have read the presentation, and it was
just brilliant.  The realization however soiled the entire
codebase (this is my personal opinion .. of someone who is neither
professor nor have some really good "hammers" out there, to earn
the merits to say something like this, that is to say).

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Greg Minshall wrote in
 <747928.1680604649@archlinux>:
 ...
 |[NB: i'm not claiming asciidoc is the right "light-weight markup
 |language" to choose.  i don't really know.  it just seems reasonable
 |enough to me.  though, probably choosing any of the options, such as
 ...

What i always hated was the paragraph-continuation via "+".  This
looks so un-natural un-textual to me.

 |asciidoc or markdown, in the future converting to some other isn't "much
 |more" than an awk script.  [my awk-fu is still okay.]]

For example, even though it is fully bloated python i think

  --config=::
  Use the given config variable as a multi-valued list storing
  absolute path names. Iterate on that list of paths to run
  the given arguments.
  +
  These config values are loaded from system, global, and local Git config,
  as available. If `git for-each-repo` is run in a directory that is not a
  Git repository, then only the system and global config is used.

Why not simply

  arguments.

  These config.

etc?  Does not work.  This totally disturbs my text-only read flow
(imagine multiple list indentations even), so i stopped thinking
about asciidoc.  I do not know anything better, too.  (And, to be
plain, when i look at some converted manuals it looks so
artificial, that a roff manual page is not that much more cryptic
in the end, really.  Even yaml that ruby used about 15-20 years
ago is de-facto easier to read .. dependent upon the content, of
course.)

  ...

P.S.: sorry for cutting so many context, all the time.
All of you.  :-(  (But HTML, .. that was hard.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Unsupported nroff macros on MacOS X

[Steffen Nurpmeso]

Anthony J. Bentley wrote in
 <2866-1680590811.027...@hnc7.hhyy.epmr>:
 |Ken Hornstein writes:
 |> Let's take the example you gave where the first line for a man
 |> page that uses tbl should contain:
 |>
 |> '\" t
 |>
 |> So, my question is ... what does this mean?  I understand that \" is
 |> a comment, but I'm confused about the leading single quote.
 ...
 |As for why the 't' is there, it's strictly a manpage practice, not
 |a general roff authoring practice. man(1) peeks at the comment (man-db
 |does anyway; pretty sure mandoc doesn't need it) to determine which
 |preprocessors to run. E.g., a manual that needs the eqn preprocessor
 |would have an 'e' instead.
 |
 |Normally the person piping a document through troff would be the author,
 |and would already know that the document contains tables that require
 |the source to be preprocessed with tbl. But manuals are formatted by
 |users; the man(1) program doesn't know the document features in advance,
 |and wants to avoid running preprocessors unless they're necessary.

   For example, newer man(1)s read the first line of the manual and
   check for a syntax <^'\" >followed by concat of [egprtv]+ (and in
   fact  *join in* $MANROFFSEQ environment [egprtv]+)
while getopts 'egprtv' preproc_arg; do
case "${preproc_arg}" in
e)  pipeline="$pipeline | $EQN" ;;
g)  GRAP  ;; # Ignore for compatibility.
p)  pipeline="$pipeline | $PIC" ;;
r)  pipeline="$pipeline | $REFER" ;;
t)  pipeline="$pipeline | $TBL" ;;
v)  pipeline="$pipeline | $VGRIND" ;;
*)  usage ;;
esac

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: nmh 1.8RC2, xlbiff, and $HOME

[Steffen Nurpmeso]

Ralph Corderoy wrote in
 <20230131181958.1cfb121...@orac.inputplus.co.uk>:
 ...
 |But if HOME is empty we do not know their intent so to ignore it and use
 |pw_dir may not be what they think will occur.  The wrong profile could
 |be read or the wrong .netrc used, upsetting the user.

By the way my mailer does:

  ? xv var HOME
  #nodelete,import-environ-first,sync-environ,notempty:
set HOME=/home/steffen

(where xv temporarily enables *verbose* mode).
If i recall correctly you, once you were still taking part, have
been dissatisfied with TOCTOU issues for TMPDIR for which the same
holds:

  ? xv var TMPDIR
  #default-value,import-environ-first,sync-environ,notempty:
set TMPDIR=/tmp

So i (fwiw) are in the camp

  $ HOME= s-nail -R#
  s-nail: Environment variable must not be empty: HOME
  s-nail: There are messages in the error ring, manageable via `errors' command
  #?0!0/NONE#ERROR|:/dev/null? var HOME
  set HOME=/home/steffen
  #?0!0/NONE#ERROR|:/dev/null? xit

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: graphical mail reader for one-off use

[Steffen Nurpmeso]

Ken Hornstein wrote in
 <20230713143447.b814036...@pb-smtp20.pobox.com>:
 |>Thanks Ken!  I'll be giving this a try!  (I would have "just tried it
 |>myself", but I don't have any modern readers installed!  Small point
 |>of pride, until now.  :-)
 |
 |One note: you MIGHT have to have Thunderbird configured properly as a MUA
 |to do this (I already had this done).
 |
 |>Huh.  ".eml".  I've spent years using and part-time developing email \
 |>clients
 |>and servers, and never heard of that extension.  Good to know.
 |
 |I am not sure those things are standardized, but I just Googled "eml
 |file extension" and it seems kind of common (some results suggested it
 |originally came from Outlook), and I had personally seen that for a
 |while.  Right or wrong, Apple had decided in the transition to OS X that
 |"file types" were determined by the filename suffix since you didn't
 |really have resource forks so the way things that use the OS facilities
 |to determine the file type will look at the file suffix.  I only mention
 |this to say that I don't know how Thunderbird will determine a file type
 |on other platforms.
 |
 |I was curious and did a little bit of digging.  New MIME types
 |that are registered with IANA can include a suggested file extension.
 |message/rfc822 predates that registry and the original MIME RFCs do not
 |specify a file extension for that type.  The message/global MIME type
 |(a RFC822 message but with UTF-8 everywhere) has a suggested
 |file extension of ".u8msg", which I have never personally seen "in
 |the wild" anywhere.  ¯\_(ツ)_/¯

I added .eml to the mime.types of the MUA on 2013-05-25 stating

  +#  is a good place to look for a 
rather
  +# complete list of IANA registered and unregistered MIME types.

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)

Re: Symbolic link to mhmail

[Steffen Nurpmeso]

Thomas Dupond wrote in
 <7c469bc6-ad63-48c0-9afb-0d6c06d5d...@dupond.eu>:
 ...
 |Wow thanks for this fix! Being able to use nmh as a mailx replacement \
 |in Debian will be very useful to me.

Sheer unbelievable.
New version up in summer, will be called "Mountains O' Things".
And has!  (More than three years of development.)

--steffen
|
|Der Kragenbaer,The moon bear,
|der holt sich munter   he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)