Re: Message security; protected header fields

[Derek Martin]

On Fri, May 10, 2024 at 02:16:12AM +0200, Eike Rathke wrote:
> Hi,
> 
> On Thursday, 2024-05-09 19:15:59 -0400, Derek Martin wrote:
> 
> > Probably fine for preventing casual eavesdropping, but for genuinely
> > sensitive applications, should not be considered good enough, unless
> > I'm missing some important detail...
> 
> If you can't trust but need to, then verify. The fingerprint over
> a trusted channel. This has been part of PGP since the beginning.

Indeed, but that's what I've been saying.  The use of autocrypt (or
similar features) downplays the need for this, and encourages naive
users to skip this important step.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Wed, May 08, 2024 at 03:49:12PM +0200, Werner Koch wrote:
> Hi!
> 
> Thanks for the summary.  I fully agree add these 2 cents:

Thanks.

> In particular using a fixed subject is not going to work in any real
> business because you are not able to ignore mails.  For my part, I even
> use a auto-responder to tell that mails with a three-dot subject are
> ignored.

Indeed.

> There is a simpler method than autocrypt to initially convey a key.  If
> you can't MIME-attach it, include your key in the signature (gpg's
> --include-key-block).  This is what S/MIME does for decades.  If you
> don't have the recipient's key (i.e. no Web Key Directory), signing the
> first message allows the recipient to reply encrypted.

This is fine, though AFAICT it still suffers from the same problems as
autocrypt:  

 - trusting authentication data from an unknown/unverified source
 - MITM

[Arguably these are really the same problem.]

Probably fine for preventing casual eavesdropping, but for genuinely
sensitive applications, should not be considered good enough, unless
I'm missing some important detail...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Mon, Apr 29, 2024 at 10:53:38PM +0200, Steffen Nurpmeso wrote:
> Derek Martin wrote in
>  |> 1. https://github.com/autocrypt/protected-headers
>  |> 2. https://datatracker.ietf.org/doc/draft-ietf-lamps-header-protection/
>  |
>  |Neat.  But this feature seems like a misfeature, making you
>  |immediately susceptible to MITM.

Hopefully it was clear from context that I was talking about
Autocrypt, not header protection, despite botching the quoting.

> I *wholeheartly* agree!  S/MIME is so much better by concept!
> This is why i like the new approach most PGP people now use, in
> that they use a signed MIME multipart which includes the public
> key as an attachment.

But... no...  This isn't better.  In either case, the problem is that
the source of the trust data--the e-mail itself--is untrusted.  You
have to have some way to verify the identity and signature of the key
out of band.  If you don't, you can guarantee that only the person who
sent you the key can read what you write, but you can't guarantee the
recipient is who you think it is.

> For PGP there really should be better (ie: TXT-based; or like so)
> SMIMEA/OPENPGKEY DNS entries, because what else one can have?
> WKD, and HKPS.  I (and many others) use OpenPGP: and point via
> https:// --- which is totally absurd given that the entire HTTPS
> aka TLS community as it is of today uses CA pools that is based
> upon commercial supermans.  No.

The trust model that OpenPGP implements is actually quite good--it
lets users establish the web of trust, and it also kind of lets you
"rate" your trust of a key, and then you can use that to make a
decision about how sensitive the info you're sending is, and whether
or not the trust level you have of the key matches it.  The problem
is, in practice, hardly anyone uses encryption at all, and of those
that do, too many use it improperly.

Unfortunately, using encryption properly and effectively is rather
unavoidably inconvenient--it requires thought and care.  Surely that's
why it has not been, and probably never will be, widely adopted. :(
The kinds of solutions like DNS records are sort of OK, if you only
care about basic privacy, but ultimately all of those require trust in
a third party--unless you run all of your infrastructure yourself,
which most people can't. But That's not good enough for applications
where encryption is actually essential, and any feature designed to
make it more convenient can only weaken the protection it provides.
For example, your ISP can hijack my DNS records if you're using their
DNS servers.  MITM is a huge problem that OpenPGP solves, but only if
you use it properly.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 26, 2024 at 06:45:57PM +0200, ilf wrote:
> The Autocrypt project worked on a draft for "Protected Headers for
> Cryptographic E-mail" [1]. That became the IETF draft "Header Protection for
> Cryptographically Protected E-mail" [2]. draft-ietf-lamps-header-protection
> is an Active Internet-Draft of the LAMPS WG, a "Proposed Standard" and it is
> on track to become an RFC.
> 
> 1. https://github.com/autocrypt/protected-headers
> 2. https://datatracker.ietf.org/doc/draft-ietf-lamps-header-protection/

Neat.  But this feature seems like a misfeature, making you
immediately susceptible to MITM.  It encourages users to forgo
establishing the trust of the keys so received.  What's to stop me
from sending you a forged e-mail that appears to be from someone else,
with an address and public key that I control?  Or, if I control
their/your local mail server, just replacing the key they gave with my
own?  Part of the point of using PGP/GPG is that you have taken the
time to verify the identity and key signature of the person presenting
the key, to prevent MITM attacks and similar.  This seems tailor-made
to encourage less savvy users (or really everyone) to do exactly the
wrong thing.

If it's not worth your time and effort to verify your recipients'
identity and keys, then encrypting your message probably isn't as
warranted as you think it is--relying on your recipients' trust in
their provider and ensuring both ends use SMTPS is almost certainly
enough (if they don't, get some that do, and if you can't trust them,
Autocrypt makes you susceptible to MITM).  And if you do take the time
to verify keys, then autocrypt is entirely superfluous.  If neither is
true, then maybe you need to re-evaluate how you are communicating, or
perhaps whether you should be communicating at all.

> Derek Martin:
> > Yeah unfortunately, as Kevin admitted, this feature was never discussed
> > here when it was implemented
> 
> Within the Mutt project, both Autocrypt and protected headers were
> discussed. I filed an issue in 2018 [3]. It was mentioned on IRC [4]. It was
> mentioned on mutt-users [5]. 13 people participated in the discussion on
> GitLab, among them an author of the current IETF draft RFC.

That's all great except I don't follow any of those mediums, and none
of those mediums are the established one for discussing the
development of new/proposed functionality in Mutt.  This list is.
AFAIK the IRC channel was meant to be supplementary, not supplantory.

Finally, to succinctly summarize my "C's" on the RFC, so that you need
not read through the whole thread:

1. Subject protection is superfluous; instead use a placeholder
   subject and put the "real" subject in the body of the message.
   Every mail client that supports encryption already supports this,
   no new functionality required.  This achieves both secrecy and
   tamper-proofing.

2. As far as tamper-proofing the headers, I personally would prefer
   putting the signature of the headers in a separate header (which
   obviously can't be included in the signature), because it doesn't
   require bloating the contents of the e-mail by doubling the space
   required to convey them (on top of any signature), and has no real
   ramifications for clients that don't support the feature (other
   than obviously they don't benefit from the signature).  However, a
   separate encrypted MIME part will also work.

3. If you want to actually hide the headers, or a subset of them, a
   separate encrypted MIME part is the cleanest way to go.  This again
   achieves both secrecy and tamper-proofing.

These all satisfy the requirements but avoid the cases of duplicate
headers or other parser bugs potentially causing headaches for less
well-implemented clients.  Aside from the need to document them, so
that clients can implement the feature compatibly, they don't even
really require new standards...  They don't introduce new behaviors
that are uncommon in existing clients.   It is extremely unlikely that
any mail client which supports arbitrary message headers and MIME
parts (i.e. pretty much all of them) won't either be able to support
these implementations, or happily ignore them.  The existing design
DOES introduce duplicated/shadowed headers in places where they aren't
commonly, so bugs are--not guaranteed--just more likely.  It's really
that simple.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 26, 2024 at 01:19:44AM +0200, Alejandro Colomar wrote:
> Yeah, most people don't even use PGP

Exactly.  On top of that, you have to use one of the small handful of
clients that support it, and then (for at least some of those) you
have to turn it on.  Like I said, virtually no one.  Or more
precisely, an infinitessimally small percentage of e-mail users.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Thu, Apr 25, 2024 at 11:07:30PM +0200, Alejandro Colomar wrote:
> While I don't have proof that no clients have major breakage with these
> header fields, I can say that the most important ones don't have major
> breakage.

Who are you to decide what the most important clients are?  If my
favorite mail client is impacted, then that's the most important
client of all... TO ME.

One of Mutt's longstanding core development principles is to stick to
the robustness principle (Postel's Law).  In particular "be
conservative in what you emit."  This is not that.  Kevin is the
maintainer, so he can do what he wants, but I think accepting this
feature violates Mutt's original development philosophy.

> mutt(1) has been protecting many header fields for several years already
> (not by default, but you could enable it), and nobody has ever reported
> a bug, did any?  The night sky is still up there and didn't fall on our
> heads.

Virtually no one uses this feature, so again, the lack of evidence is
all but meaningless.  But sticking to good software design principles
does matter, even in the absence of documented cases of nonconfirmity
to good software design principles causing actual problems.

> That's a bit contradictory to another section of the same RFC:

I've read the RFC, I know what it says.  I already noted in previous
messages that adding the headers is ALLOWED by the RFC.  But it
historically hasn't been something that is generally done, and just
because something CAN be done doesn't mean it SHOULD.  Any time you
change the input data to a program in an unexpected way, a bug is the
LIKELY consequence.  I gave some legitimate examples of the sorts of
bugs that are LIKELY due to this implementation due to its
implementers not expecting that behavior.

There have been two alternatives suggested that do not suffer from
this problem, or any other as far as I can tell.  Either of those
would seem obviously better, and while I have no use for them, I would
have no objection to them.  They don't even require a new standard
(although you certainly should draft and submit one anyway)--all the
specification required is already part of either RFC 822 or the MIME
standard RFCs...

You probably would need to register a new MIME type with IANA or
whatever, to distinguish the signature from that of the message body,
but I imagine that's easier than writing a whole new RFC and getting
it approved.

> I think RFC 2045/2.4 specifically allows that as an extension.

Allows it, technically, maybe... but it does not DEFINE it.

Particularly with regard to the content of messages it generates, Mutt
implements standards.  Or at least historically it did.  This is not
that.

> And the draft that has been mentioned here is the document for
> documenting the extension.

But again, it's not a standard, it's just a specification.  A standard
is a specification that has been reviewed and accepted by either a
designated standards authority, or by the associated community
generally.  This document is neither.


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Thu, Apr 25, 2024 at 10:22:16PM +0200, Steffen Nurpmeso wrote:
> You talk to the wrong person given that other people added that
> mechanism and put it into practical use.

Yeah unfortunately, as Kevin admitted, this feature was never
discussed here when it was implemented, so those who care about any of
that are having the conversation now.

But that doesn't matter... I was specifically responding to YOUR
point, which was a logical fallacy.  You seemed to be making a point
that implied that if you aren't aware of clients that are negatively
impacted, then it doesn't matter.  That's bad logic.

There's a difference between bugs caused by the implementer of the
receiver failing to correctly implement what is normal and expected,
and bugs caused by the implementer of the sender intentionally
implementing behavior that is outside what is normal and expected.
The former affects only the user of that software.  The latter affects
everyone else but the user of that software (at least potentially).
Responsible developers should strive to avoid both, but the latter
clearly has a more serious impact on more people, and exactly the
wrong people.  It is the whole reason the robustness principle exists.
Outlook is famous for this sort of thing, and it's one of the reasons
people hate Microsoft/Outlook.  

> (And regarding this -- i think this works out fine.  Except for
> dumb clients like mine which do not get the thing and do not map
> those headers into the place of the main ones.  Yet.)

You have no proof that this is true, among the hundreds if not
thousands of e-mail clients that exist.  That is the nature of
undefined behavior.  There's nothing special about that term in the
context of ISO C or otherwise.  And since there isn't any sort of
official standard, it's probably rather unlikely that the majority of
clients will ever addopt this.  You can't call them dumb for not
implementing a standard that doesn't exist.

> You can expect anything in it is what is said.  Of course, MUAs
> which do not understand maximally visualize those header lines in
> addition to the main ones (what mine does), or totally ignore them
> (what i expect graphical ones to do).  That is not "undefined
> behaviour".

These are not the only possibilities, since again, the behavior is
undefined.  Headers could be incorrectly repeated (which could lead to
additional bugs, since that also is outside the spec), the "wrong"
version of the header could be the one the client uses.  Or the
structure you're using to hold the headers could overflow, causing
memory corruption and/or crashes.  Etc..  Those things might be
unlikely or impossible to happen in the absence of the undefined
behavior.  It's hard to be robust against problems you didn't forsee,
on account of them being well outside the norm.

Undefined behavior is... undefined.  When the input data deviates from
what is expected, any variety of resulting bugs is LIKELY.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Wed, Apr 24, 2024 at 01:07:10AM +0200, Steffen Nurpmeso wrote:
> Sirius via Mutt-dev wrote in
>  |I would worry less about the users and more about breaking clients. The
>  |method of "be liberal about what you receive and conservative about what
>  |you send" is apt. Being standards-compliant is safe.
> 
> Ah, do you know about any one client which fails on headers it
> does not understand,

Absence of evidence fallacy.  For this to be a non-concern (logically
speaking) you would need to prove that NO client has this problem.
Proving a negative is not always impossible, but given the number of
clients in existence, it's pretty impractical.

Now, how that leaves you IS NOT with conclusive proof that you should
not do it this way... but rather a strong suggestion that IF you can
find a good alternative that doesn't have the same potential weakness
(nor other worse tradeoffs), choose that instead.

But I'm repeating myself now.

> ok i have reread 2045 and it says [...]

Largely irrelevant, because of what it does NOT say...  For instance,
it does not describe what the behavior should be if standard RFC 822
headers appear BOTH in a mime header block AND in the actual message
headers.  This is what's known as undefined behavior.  Therein lies
the path to non-interoperability--which Mutt intends (or, at least has
historically intended) to avoid.  I've already described how such a
problem might arise in a previous message.  Avoid forseeable
interoperability problems when possible.

Also, what the RFC does explicitly state is that headers in the MIME
header block that do not begin with "content-" "CAN HAVE NO MEANING"
[emph. mine], and "may be ignored."  It gives no indication that there
would be an exception to those conditions for headers that are
explicitly defined by RFC 822 (or its successors).  Therefore any
processing of them that you do is at best ambiguous, i.e. again,
undefined behavior, and at worst violates the spec (because processing
them gives them meaning that the spec says they can not have).

Which of course isn't to say that the standard can't be extended or
modified; but if you want to do that, then you should actually draft
the standard extension and get it approved, well before asking clients
whose central tenets include complying with standards (as Mutt's do)
to implement such extensions.  The reasons to do that are to establish
whether the relevant community even values the extension, and whether
better alternatives may exist or be found.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 10:44:18AM -0400, Derek Martin wrote:
> On Fri, Apr 19, 2024 at 10:41:58AM +0800, Kevin J. McCarthy wrote:
> > However, saying that mutt adds those headers by accident or as a bug seems a
> > bit uninformed.
> 
> Fair enough.

But--and I s'pose I may have just missed it--I don't recall seeing any
discussion of this feature on this list.  Learning of such things is
precisely why I'm on this list, so... I'm not sure whether it's better
(from my perspective) that I missed it, or if it never happened. =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 10:10:13PM +0200, Alejandro Colomar wrote:
> Hi Derek,
> 
> On Fri, Apr 19, 2024 at 03:41:40PM -0400, Derek Martin wrote:
> > On Fri, Apr 19, 2024 at 09:05:23AM -0700, Will Yardley wrote:
> > > It's odd to me that, since OpenPGP and S/MIME both support MIME
> > > encapsulation that the draft standard wouldn't use a separate MIME part
> > > to handle the protected headers vs. stuffing it at the top of the
> > > message body, which just seems kind of kludgy at best.
> > 
> > This also seems like a perfectly cromulent approach, and again better
> > than the proposed one which puts nonstandard headers in a place where
> > no standard says they can be. [Or perhaps the correct wording would
> > be, "...which nonstandardly puts standard headers in a place where no
> > standard says they should be."]
> 
> How would a program differentiate between a body whose contents are
> header-field-like?

You're missing the point:  All software has bugs.  This would be a
bug, but it's forseeable that mailers would have this bug.  For
example, if the header parser you implement differentiates where to
put the headers it's parsed based on what the header is--i.e. standard
header vs. mime header, it may add the header to the wrong object, and
now you have two subjects, two from headers, etc...  They are both
headers, valid headers, that should be treated as headers--it just got
the context wrong based on the name.

In most cases this bug would be completely innocuous--and perhaps even
beneficial (e.g. because it simplified the parser)--because mime
header blocks typically should not have RFC 822 headers in them.
Except now you've added a reason why they might.  Depending on what
the borked client does next, this could inject further broken e-mail
into the thread causing problems for everyone on it.

Does this bug exist anywhere?  Does any similar bug exist anywhere?  I
don't know, but it's forseeable that such bugs may exist.  Should you
always avoid implementing functionality that might interact with a
bug?  No... but if you can find a better design that is far less
likely to interact with bugs, you should probably use that instead.

> already a decent time frame that other projects can do the same thing.
> Moreover, Memory Hole, later autocrypt, and now the IETF proposed draft,
> all use this mechanism.  So we can say it's de-facto standard, even if
> it's not yet standard.

I don't think you can... I've never heard of Memory Hole, nor until
this conversation had I heard of autocrypt.  Outlook's long line
parsing can be called a defacto standard, because it's used by
millions, perhaps even billions of users every day.  These ain't.

And even if they were, just like Outlook's long line parsing, defacto
standards are often very bad standards.

And also, autocrypt DOES NOT use existing standard headers, so it
doesn't put standard headers in the MIME block.  That's an important
distinction which would not trigger any bug like the one I described
above.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 09:58:42PM +0200, Alejandro Colomar wrote:
> I think these days MIME is not so frowned upon as it once was.  But you
> have a point.  patatt(5) actually implements an idea like yours for
> signing patches including header fields, precisely for avoiding MIME.

To be clear, I love MIME.  I'm not trying to avoid it, and I think
Will's suggestion using MIME is absolutely fine.  I'm suggesting that
the existing implementation pollutes MIME, which should be avoided, as
it presents an opportunity for existing MIME-compliant mail readers to
be confused by RFC 822/2822/5322 headers being present in MIME header
blocks, which is not something any standard even hints at.  I'm not
saying any such clients exist, but given the vast array of clients, it
would be a challenge to prove that none do.  It's just bad practice
and should be avoided.

[The MIME spec says non-MIME headers can be present and should be
ignored, but it doesn't address handling duplicates of standard
headers, and it likely wouldn't occur to past implementers of MIME
that someone would want to do this... it's essentially undefined
behavior.  In most cases it probably should be fine, as long as
implementers were careful; but it's going to depend on how message
parsing / MIME header block parsing is implemented in each client.]

For the record, if this is important to you, I'm also not saying you
shouldn't implement it.  Kevin hinted he wouldn't be too keen on
working on adding this to Mutt, which is pretty consistent with Mutt's
history--maintainers have always leaned pretty conservative about
accepting patches, and for good reason.  And especially so as Mutt is
in maintenance mode.  But there's no reason you can't create them and
make them available to anyone who wants this.  That's something that's
also been a part of Mutt development for decades--I and many others
have maintained patches for Mutt for extended periods, some of which
eventually made it in, many of which did not. I'd just note that, as
it would be a subset of people who use Mutt and use Mutt's encryption
features--already a fairly rarified group--I suspect the audience for
such patches will be quite small indeed...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 09:30:27PM +0200, Steffen Nurpmeso wrote:
> Derek Martin wrote in
>  <20240419191717.ge2...@bladeshadow.org>:
>  ...
>  |Secondly, there is a standard mechanism for adding non-standard
>  |headers to e-mail:  use the string "X-" before the thing, and add it
> 
> Not anymore since RFC 6648.

Eh... as far as I can tell that RFC is routinely ignored.  But fine,
just omit the "X-" and make sure the field names are unique.  This is
still allowed by RFC 5322.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 09:05:23AM -0700, Will Yardley wrote:
> It's odd to me that, since OpenPGP and S/MIME both support MIME
> encapsulation that the draft standard wouldn't use a separate MIME part
> to handle the protected headers vs. stuffing it at the top of the
> message body, which just seems kind of kludgy at best.

This also seems like a perfectly cromulent approach, and again better
than the proposed one which puts nonstandard headers in a place where
no standard says they can be. [Or perhaps the correct wording would
be, "...which nonstandardly puts standard headers in a place where no
standard says they should be."]

MIME header blocks are for MIME-specific metadata; even if no mail
clients actually break due to this, it still feels gross.  Yes, the
MIME spec says other fields can appear there and should be ignored (so
long as they don't start with "content-") but having standard message
headers in multiple places unexpectedly still runs the risk that
insufficiently carefully implemented clients will get confused.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Thu, Apr 18, 2024 at 08:38:29PM -0400, Kurt Hackenberg wrote:
> Signing header fields sounds reasonable, but I don't entirely like an
> implementation that puts a copy of them in the message body, to be covered
> by GPG.  I'd prefer something more direct, that signs headers without
> copying them or modifying the message body.

This makes a lot of sense to me (assuming that this feature has
genuine practical value, of which I remain largely unconvinced).  

First off:  Is there an actual RFC for the existing protected header
scheme?  If not, someone really ought to actually draft one and submit
it, so that more people than just a handful of interested users of
a small selection of specific mailer software could provide comments
on the Request For Comments...

Secondly, there is a standard mechanism for adding non-standard
headers to e-mail:  use the string "X-" before the thing, and add it
to the normal message headers.  To the extent that this feature is
useful, it would be just as useful for non-encrypted messages as for
encrypted messages.  You could do something like this:

X-protected-header-key:  0x1234567...  # probably unnecessary, but
might be interesting for confirmation?
X-protected-headers: (A base64-encoded string representing the precise
and complete contents of the headers which were signed)
X-protected-headers-signature:  (the actual signature of the data made
by the specified key)

The user could verify the signature of the signed headers, so long as
the key is available and trusted.  The mailer could display and/or
visually confirm the signed headers against the standard ones,
ignoring white-space differences, etc..

This is not very well fleshed out, but it already strikes me as better
than what Mutt currently does: 

  - it does not require even having MIME parts in such a message.
  - I don't love what Mutt is currently doing--it seems it has the
potential to break existing clients which are not expecting
message headers in the MIME header blocks.  A well-behaved client
probably shouldn't break, but as we all know, not all software is
well-behaved.  All mailers are expected to accept/ignore X-*
headers.

Also FYI, I removed the neo-mutt list from this post because I'm not a
member and I don't cross-post; and also you probably don't need to
include Werner separately, he's a longstanding member of this list. :)


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 10:41:58AM +0800, Kevin J. McCarthy wrote:
> However, saying that mutt adds those headers by accident or as a bug seems a
> bit uninformed.

Fair enough.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Thu, Apr 18, 2024 at 08:16:15PM -0400, Derek Martin wrote:
> The message interception scenario is possible, but I think highly
> improbable, especially for the sort of people who are using Mutt and
> encryption--savvy users.  It requires the attacker have superuser
> access to the mail system somewhere between you and your genuine
> recipients, AND either be known to your recipients, or your recipients
> must have no idea who should be on the message.  The attacker needs to
> be able to prevent the delivery of the original, to have time to
> inject the bogus message.  And your recipients need to already have
> the attacker's public key and trust it, or be set up to automatically
> download and use untrusted public keys...  

Also, FWIW, I've been using Mutt and encryption for almost 30 years,
and I haven't heard of a single case where the way Mutt handles
encrypted mail caused someone embarrassment or loss.  Not to say there
haven't been any, but... at least if there were, not high enough
profile to make it HERE.

And to be honest, it's been my experience that your recipients are
much more likely to leak the stuff you sent them encrypted, through
their own stupidity/carelessness than because of any flaw in Mutt.
And yes, that has happened to me.  Be very careful about relying on
encryption to save you.  You're probably almost always much better off
just not saying whatever you thought you should encrypt.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Fri, Apr 19, 2024 at 01:59:57AM +0200, Alejandro Colomar wrote:
> BTW, now that I remember, while developing these things for neomutt(1),
> I found that mutt(1) has a bug (?) by which it does actually protect
> some header fields precisely in the way that I implemented them in
> neomutt(1), with the difference that mutt(1) does it on accident.

I don't know whether or not it's intentional, but it does seem to be a
bug to me...  The message headers do not belong in the MIME headers
section.  But also note that Mutt doesn't do anything with them,
including display them (at least not here).

>   $ mutt -v | head -n1
>   Mutt 2.2.12+68 (841caf1c) (2023-10-25)

Yeah, mine is a bit older. =8^)


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Thu, Apr 18, 2024 at 11:59:29PM +0200, Alejandro Colomar wrote:
> Protecting the recipients and the in-reply-to doesn't mean hiding it.
> It means providing a copy inside the signed part, so that it can be
> verified against tampering.  It's not about encrypting them.

You can already do this in mutt by using a script as your editor to
parse the headers, add the info you want to the body, and then edit
your message.  But it doesn't really guarantee what you want because
as I already pointed out, the message headers may differ from the
actual recipients for a variety of reasons (Bcc, forwarding, mailing
lists, etc.).  It may indicate a difference--it does not guarantee
that the difference is caused by tampering.  It most likely is not, so
it will most likely mislead you.

The message interception scenario is possible, but I think highly
improbable, especially for the sort of people who are using Mutt and
encryption--savvy users.  It requires the attacker have superuser
access to the mail system somewhere between you and your genuine
recipients, AND either be known to your recipients, or your recipients
must have no idea who should be on the message.  The attacker needs to
be able to prevent the delivery of the original, to have time to
inject the bogus message.  And your recipients need to already have
the attacker's public key and trust it, or be set up to automatically
download and use untrusted public keys...  

> BTW, mutt(1) is already incompatible with e.g. thunderbird(1) regarding
> the Subject.  That's precisely why I wanted some agreement on this
> outside of just neomutt(1).

How so?

> > 2. Many of these violate existing standards, or at least have no
> >standard.  Standards exist for a reason--if you don't follow them,
> >other people who don't happen to use the exact same client as you
> >won't be able to interoperate.
> 
> I don't think so.  Again, I don't think you've understood the points.

For instance, if you rely on the in-body headers when they differ from
the message headers, the other recipients' response behavior will
differ from yours, because that is not how e-mail works.  The
standards dictate that e.g. the reply-to header--not some random text in
the message body--is the address that e-mail clients should send
replies to.  So yes, it violates the spec.

> Have a lovely night!

You too!

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Message security; protected header fields

[Derek Martin]

On Thu, Apr 18, 2024 at 06:37:50PM +0200, Alejandro Colomar wrote:
> Hi mutt(1) and neomutt(1) developers!
> 
> I reported around a month ago a couple of security vulnerabilities to
> neomutt(1), but which are also present in mutt(1) and every MUA
> (probably, I didn't do an exhaustive research).

While I don't presume to speak for Kevin, or anyone with commit
access, here are my thoughts:

1. None of the issues you listed are security vulnerabilities.  I'm
   also fairly sure that anyone sophisticated enough to use Mutt AND
   make regular use of encryption is not going to be confused by any
   of the display-related issues you pointed out.  If you are going to
   use encryption, then it's incumbent upon you, the user, to
   understand how it works, what it protects, and what it does not
   protect.  It's definitely an advanced use case.
   
   a. If you don't want to expose the "real" subject--don't.  You have
  full control over what you put there.  Likewise, you can also
  write whatever you want in the body of the message to address
  that.

   b. There are valid reasons (e.g. Bcc) why the message may be
  encrypted to someone not on (your copy of) the headers.  By and
  large, as the recipient, that's none of your business.
  Presumably, the sender knows what's happening and that's what
  matters.

   c. Pretty much the same thing with a listed recipient to whom the
  message is NOT encrypted.

   d. The blank lines are for readability--a display-only concern--and
  displaying them has no actual impact on security.  If you have
  reason to do something with the actual encrypted body other than
  neatly displaying it for your reading pleasure, process the
  actual message body outside Mutt in whatever manner you care to.

   e. "Hidden" recipients:  If you want to hide your recipients from
  each other, just send a separate message.  I haven't ever needed
  to but presumably you can use Mutt's command line interface to
  script this, so it's probably actually pretty trivial to do that
  already.

   f. "phishing protection" -- First, in the context of encrypted
  messages, this doesn't seem like a legitimate concern.  You're
  going to know who your recipients are, and their keys, and this
  won't be a factor.  But recipients' (or senders') addresses
  which are not 8-bit clean need to be encoded anyway, according
  to RFC 6531, so just looking at the headers will already reveal
  that.

   g. Protecting the recipients is problematic for potentially several
  reasons--it prevents people from interacting normally with
  threads and their recipients.  The SMTP envelope needs at least
  the recipient you're actually sending to in that specific SMTP
  session, and the MTA will add the recipient's address in a
  Received header, so hiding that at least is pointless.  But if
  Mutt added these features it would be the only client to do so
  (AFAIK) making it very difficult for anyone using any other
  client to deal with.  You couldn't simply "reply-all" to, well,
  reply to all...  And again, you can just send separate messages.
  And if you really don't want your recipients to know about each
  other, then you shouldn't be sending them the same message
  anyway!  Avoid any possibility of embarrassment or worse--send
  separate messages.

   h. The rest are just minor display issues--preferences--and have
  nothing to do with security whatsoever AFAICT.

2. Many of these violate existing standards, or at least have no
   standard.  Standards exist for a reason--if you don't follow them,
   other people who don't happen to use the exact same client as you
   won't be able to interoperate.

3. Probably most importantly, Mutt is basically in maintenance mode,
   i.e. no new features.  So none of this is likely to get
   implemented, even if one of the developers happened to agree with
   you.
   
Personally, I would not be in support of the vast majority of what you
suggested.  Maybe the "hidden recipient" for Bcc, but again, avoid the
whole problem and send a separate message.  But the bulk of what
you're suggesting is a ton of work and extra complexity for really
very little or no genuine benefit, and potentially significant
interoperability problems.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Ctrl-C / SIGINT behavior

[Derek Martin]

On Thu, Dec 14, 2023 at 12:22:22AM +0100, Vincent Lefevre wrote:
> > Mutt is a user application that already provides two methods of
> > exiting the program "nicely" with two different behaviors:  Save state
> > or don't.  CTRL-C/SIGINT allows for a third way to terminate the
> > program when it is behaving in an incorrect/undesirable way, when for
> > example you're not able to get to a menu, or Mutt is failing to
> > process menu selections for whatever reason.
> 
> OK, but that's not the only use of Ctrl-C. It can also be used to
> interrupt an ongoing operation, such as a search/limit, but it may
> happen that the time the user hits Ctrl-C, the operation has just
> finished, so that instead, Ctrl-C will attempt to terminate Mutt.
> One may also hit Ctrl-C by mistake.

Fair--ideally those would be two different functions mapped to
different keys.  I can't immediately think of a way to divorce them
without rewriting Mutt to process UI input in a separate thread
(eww)...

> > Mutt already prompts to allow for the possibility that you've hit
> > CTRL-C by mistake.  When you haven't, clearly the intention is to
> > terminate immediately, so that is quite sensibly the default, whereas
> > whatever you configured for OPT_QUIT doesn't necessarily apply in this
> > circumstance, since the assumption there is that you are exiting under
> > normal and expected circumstances.
> 
> There could be a new option, say OPT_SIGINT, which could have the same
> default as the current behavior. So, for users who do not change this
> option, there would still be the same behavior with the introduction
> of "query_quadoption (OPT_SIGINT".

Sure, I don't think this is a crazy suggestion either.  As you
probably recall, I'm generally averse to adding new options.  The
number of options Mutt already has drastically increase its learning
curve and make it hard (tedious, time-consuming, error-prone, etc.) to
configure the way you want it, and every option is an increase in
complexity, a potential risk for maintenance/testing, etc..

That said, this may well be straightforward, and the pragmatist
in me thinks that the ship has already sailed on the "too many
options" problem.  ¯\_(ツ)_/¯

> > If not for this distinction, you could simply ignore SIGINT
> > entirely.
> 
> I don't understand what you mean. I can't tell Mutt to ignore it in
> this particular case.

Essentially I was saying that--arguably--Mutt shouldn't need a third
mode of exiting, so

  signal(SIGINT, SIG_IGN); /* but use the POSIX interface */

...EXCEPT that there IS a distinction between how SIGINT is used
compared to quit/exit, so there is value in the third mode.  [And of
course if you did that, you would need to address the other uses of
SIGINT somehow...]

> >--particularly when you
> > obviously already realize this is a mistake you sometimes make--and
> > as I'm certain you are aware, there are already solutions possible to
> > help you deal with that (remapping Mutt's keys, remapping SIGINT in
> > your terminal, etc.).
> 
> First, remapping Mutt's keys can't affect the intr character.

No, but it CAN make Copy Message (or whatever you have bound to
SHIFT-C) bound to a different key that prevents you from mixing up
SHIFT-C and CTRL-C...  And you can still remap SIGINT to another key
in your terminal.

BTW FWIW, I think the reason I've never had this problem is that I
don't use copy message all that often, and when I do the folder it
suggests is ALWAYS wrong.  So I can't ever just hit enter, and have to
actually pay atention to the prompt to correct it.  So here again,
this comes down to personal habit and usage patterns.  I suspect the
majority, and probably even the vast majority, of Mutt users are in a
similar boat as me.  But it took even you almost 30 years to complain
about it. ;-)

> And changing the standard Ctrl-C for SIGINT would be more
> error-prone, with no simple way to restore the behavior, e.g. after
> a Ctrl-Z, or when running shell commands.

All of this comes down to personal habit and preference (e.g. don't
CTRL-Z out of Mutt--use a different terminal instead).  You can
certainly dislike the available options, but you can't argue that it
can't be done--there are several ways for you to work around this.

> > Perhaps, but since 1995, I'm pretty sure I've personally made that
> > mistake exactly zero times, nor have I ever noticed anyone complaining
> > about this in the past, here or anywhere.  That leads me to think that
> > rather than changing the current behavior--which seems already
> > sensible--you should adapt to it yourself, as I suggested above.
> 
> Sorry, but this is silly.

No it isn't, stop doing that.

> Software should adapt to the human, not the other way round.

Yeah, I agree with that as a general principle, but there are limits,
and there have to be.  An extreme, obvious example is if an app uses a
particular paradigm that is, say, new (and therefore different from
existing ways of doing that thing), the user who wants 

Re: Ctrl-C / SIGINT behavior

[Derek Martin]

On Wed, Dec 06, 2023 at 03:10:11PM +0100, Vincent Lefevre wrote:
> The behavior of SIGINT (typically generated by Ctrl-C in the terminal)
> is not documented.
> 
> It currently asks whether to exit, with MUTT_YES as the default:
[...]
> Why doesn't it use "query_quadoption (OPT_QUIT," like  and ?

I don't think your suggestion is completely crazy, but you asked, so I
will provide the counterargument:

Mutt is a user application that already provides two methods of
exiting the program "nicely" with two different behaviors:  Save state
or don't.  CTRL-C/SIGINT allows for a third way to terminate the
program when it is behaving in an incorrect/undesirable way, when for
example you're not able to get to a menu, or Mutt is failing to
process menu selections for whatever reason.

Mutt already prompts to allow for the possibility that you've hit
CTRL-C by mistake.  When you haven't, clearly the intention is to
terminate immediately, so that is quite sensibly the default, whereas
whatever you configured for OPT_QUIT doesn't necessarily apply in this
circumstance, since the assumption there is that you are exiting under
normal and expected circumstances.  If not for this distinction, you
could simply ignore SIGINT entirely.

One could argue that your actual problem is that you aren't adequately
careful about reading and responding to prompts--particularly when you
obviously already realize this is a mistake you sometimes make--and
as I'm certain you are aware, there are already solutions possible to
help you deal with that (remapping Mutt's keys, remapping SIGINT in
your terminal, etc.).

> This would be safer in case one types Ctrl-C +  by mistake.
> I often do Shift-C +  (very quickly), but I sometimes mistype
> Shift-C as Ctrl-C, hence the issue.

Perhaps, but since 1995, I'm pretty sure I've personally made that
mistake exactly zero times, nor have I ever noticed anyone complaining
about this in the past, here or anywhere.  That leads me to think that
rather than changing the current behavior--which seems already
sensible--you should adapt to it yourself, as I suggested above.

> BTW, as mutt_query_exit() doesn't save, the message should be
> "Exit Mutt without saving?" like with , IMHO.

This seems perfectly sensible, though I also think that to anyone
who's familiar with what CTRL-C typically does on a Unix terminal (and
plenty of other places), this is already implied and obvious.  I would
note that if the issue is that you're hitting  too quickly to
read the prompt and respond to it appropriately, the wording doesn't
matter much. =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] send.c: Allow crypto operations in batch and mailx modes.

[Derek Martin]

On Wed, Nov 15, 2023 at 09:05:15PM +0100, Alejandro Colomar wrote:
> Hi Darek,

Derek.

> > So, acknowledging that this discussion is mostly academic since
> > there seems not to be anyone to maintain/support new features...
> 
> Yes, it's still academic and useful, since I plan to patch neomutt(1) in
> the same way (and neomutt(1) is still maintained).  Only problem with
> neomutt(1) is it doesn't seem to be compatible with mutt(1), in that
> `neomutt -H -` doesn't seem to work.  (I reported that as a bug to
> neomutt(1), although it seems they removed the feature on purpose a few,
> years ago, which seems confusing to me.)

Neomutt and Mutt have different development philosophies (demonstrated
merely by the existence of both, in fact).  So what's right for one
isn't necessarily what's right for the other.

> > > but at least it keeps it simple: you enable crypto in
> > > the config, you get crypto.
> > 
> > No it doesn't.  I use both encryption and unattended mail, and I don't
> > use gpg-agent or equivalent, quite intentionally, as I want the choice
> > to use encryption to be quite deliberate, requiring me to type my
> > passphrase each time I do so. Your method completely breaks me.  No
> > thanks.
> 
> You could use a different config for those unattended actions.

I'm well aware that I could, but why would I want to do that?  Let's
compare and contrast the two solutions.

Werner's solution:
 - Does not EVER affect anyone who does not want the new behavior
 - If you do want it, requires only setting one variable, once
 - The software can automatically avoid using the new behavior when
   using it would be impossible.

Your solution:
 - Affects every user who does not want the new behavior, every time
 - Forces every affected user, individually, to think about how best
   to avoid the new behavior, for their workflow and usage patterns
 - Forces negatively impacted users into one or more modes they didn't
   want:
• Starting an agent they may not have wanted to use to circumvent
  the new behavior
• Maintaining multiple config files, for different purposes
• Having to choose between multiple config files, each and every
  time the user starts Mutt, whether interactively or via script,
• etc.

Which option provides a better user experience to users, both affected
and unaffected, as a whole?  If you don't think it's VERY CLEARLY the
first one, I might think maybe you should not be designing software
features...

Werner's solution is absolutely simple, and affects people the least
who care about the feature the least, and affects users the most who
have incentive to care the most, but even then requires only a very
minimal one-time effort.  When you're adding a feature to software,
that's probably what you want, in most cases.

Your solution impacts most those who care the least about the new
feature, introducing numerous places where, in order to avoid the new
behavior, the user must do more work and may make a mistake on an
ongoing basis, when they have to maintain their config files, and when
they start Mutt, each and every time.  It's a tedious waste of time.
Worst of all, if the user ever fails to do the thing that turns on
encryption when it was needed, your solution potentially exposes them
or their data, in a way that Werner's would not.  Or rather, due to
the fallibility of humans, your solution actively makes that more
likely. I think it's fair to say it is objectively bad, for all of the
above reasons.

Now, it could be a different matter, if the feature were one that is
generally useful to most e-mail users (e.g. updating some aspect of
message generation to a new standard), and it was just a small group
of stubborn, curmudgeonly users that were affected.  That's not the
case here (even if the second part might be)--this feature is only
useful to a subset of a subset of an already small community of e-mail
users.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] send.c: Allow crypto operations in batch and mailx modes.

[Derek Martin]

On Wed, Nov 15, 2023 at 07:10:52PM +0100, Alejandro Colomar wrote:
> On Wed, Nov 15, 2023 at 04:13:14PM +0100, Werner Koch wrote:
> > Hi!
> > 
> > On Fri, 10 Nov 2023 01:41, Alejandro Colomar said:
> > 
> > > This is breaking behavior, so it needs some more justification than just
> > > the above.
> > 
> > FWIW, I am using another patch for 2 years now to send unattended but
> > signed mails.  The patch requires a new option to avoid the risk of
> > regressions.  See
> > 
> >   https://lists.mutt.org/pipermail/mutt-dev/Week-of-Mon-20220725/thread.html
> > 
> > and Kevin's follow-up.  Unfortunately I had not have the time to
> > continue working on the patch to get this or something else upstream.
> 
> Hmm.  Interesting.  I think I prefer not adding an option.  It's
> breaking behavior

So, acknowledging that this discussion is mostly academic since
there seems not to be anyone to maintain/support new features...

> but at least it keeps it simple: you enable crypto in
> the config, you get crypto.

No it doesn't.  I use both encryption and unattended mail, and I don't
use gpg-agent or equivalent, quite intentionally, as I want the choice
to use encryption to be quite deliberate, requiring me to type my
passphrase each time I do so. Your method completely breaks me.  No
thanks.

Besides which, this is Mutt's longstanding development philosophy.  It
violates principle of least surprise.  An option is required. [And
this, from a guy who hates adding options...]

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Avoid falsely uncollapsing threads for old messages

[Derek Martin]

On Wed, Sep 14, 2022 at 02:53:27PM +0200, Magnus Groß wrote:
> The thing is that even according to your definition, mutt is
> behaving incorrectly: The outgoing message saved via "record" is
> **not** a newly received message.

Yes, it is.  Which is to say, when you place a message in a mail
folder that was not there before, this is precisely the mechanism of
delivering mail to that folder, and is indistinguishable from mail
delivery.  If your MUA scans for new mail and finds messages it hasn't
indexed yet, it's effectively new mail--regardless of how it got
there.

> Could you point me in the right direction how to best solve this
> problem?  Maybe we would need to track messages that we save via
> "record", but maybe there is an easier fix.

You're essentially treating your own messages like new mail by saving
them to your incoming mail folders.  The record is not really meant to
be used that way, although people do it, to keep the thread intact in
their mail folder.

Solution: don't do that.  I get why you want to, but... you cannot
have everything.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Upcoming freeze for 2.2.0 and maintainer update

[Derek Martin]

On Wed, Jan 26, 2022 at 10:21:52AM -0800, Will Yardley wrote:
> On Mon, Jan 24, 2022 at 11:30:56AM +0100, Eike Rathke wrote:
> > On Sunday, 2022-01-23 09:47:57 -0800, Kevin J. McCarthy wrote:
> > 
> > > Thank you all for your support, and to the development team for
> > > giving me the maintainership opportunity.
> > 
> > I can't thank you enough for keeping mutt being the mailer that sucks
> > less!
> 
> +1 on this! Great work on helping keep mutt useful and relevant, and
> being a good steward, as well as helping balance all the strong opinions
> / personalities.

+1 to this. =8^)


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: $reply_prefix

[Derek Martin]

On Fri, Jan 14, 2022 at 11:05:14AM +0300, Jean Louis wrote:
>The "Subject:" field is the most common and contains a short
>string identifying the topic of the message.  When used in a
>reply, the field body MAY start with the string "Re: " (an
>abbreviation of the Latin "in re", meaning "in the matter of")
>followed by the contents of the "Subject:" field body of the
>original message.  If this is done, only one instance of the
>literal string "Re: " ought to be used since use of other strings
>or more than one instance can lead to undesirable consequences.

> It MAY, and it MAY NOT. There is no strict rule to it.

Not so.  As another poster attempted to point out, "MAY" (all caps)
has a very specific meaning in RFCs. However even in standard English,
the same meaning applies here.  The RFC names exactly one thing
that MAY be done, specifically in the context of subject lines in
replies--only that one thing (starting the subect string with the
expressly stated string "Re:") may be done... You can choose to NOT do
that thing, but this does not give you permission to do any other
random thing you like.

Secondly even ignoring that, your interpretation makes zero sense in
the context of what follows, where it discusses the handling of (quite
specifically) the "Re:" strings from multiple replies... but not
random other strings.

> Thus RFC5322 does not contribute to overall understanding. It remains
> as capricious decision by Latin language speaker who introduced it in
> the document. It does not represent international consent.

Again, no.  Latin phrases and abbreviations are present in formal
writing of most languages on Earth, and typically in formal writing
and acadamia their usage is expected and even required, in some
circumstances.  Latin is typically (still) used in such contexts
probably largely out of custom, but because it formerly was the
ubiquitous common language of acadamia, and the many such
abbreviations are short, precise, and generally commonly understood by
educated people of all nations and languages.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: $reply_prefix

[Derek Martin]

On Fri, Jan 14, 2022 at 09:10:19AM +0200, Tapani Tarvainen wrote:
> On Thu, Jan 13, 2022 at 08:20:55PM -0800, Kevin J. McCarthy (ke...@8t8.us) 
> wrote:
> 
> > I've been told other prefixes are often used in some lists, and the
> > practice is getting more common.
> 
> Other prefixes have also long been used by some non-English speakers
> and lists. 

Indeed, which is why the RFC exists.  But it has long been a design
principle of Mutt that it should emit only output which conforms to
the RFCs, in keeping with the robustness principle:  "Be lenient in
what you accept, but strict in what you emit."  When other mail
clients (or mail users) do not adhere to that, Mutt is not at fault.

I realize there is an element of "tilting at windmills" to this, but,
if you run into such cases which cause issues, your complaint should
be with tthose users or mail clients, not Mutt, since Mutt obeys the
rules.


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: $reply_prefix

[Derek Martin]

On Fri, Jan 14, 2022 at 04:27:10PM -0600, Derek Martin wrote:
> On Thu, Jan 13, 2022 at 09:25:27PM -0800, Kevin J. McCarthy wrote:
> > On Thu, Jan 13, 2022 at 11:57:37PM -0500, John Hawkinson wrote:
> > > Kevin J. McCarthy  wrote on Thu, 13 Jan 2022
> > > at 23:20:55 EST in :
> > > 
> > > > I've been told other prefixes are often used in some lists, and the 
> > > > practice
> > > > is getting more common.  Why not give users the option to adjust it, if 
> > > > they
> > > > deem it appropriate, for some lists?
> > > 
> > > The reason not to is that the knob encourages the proliferation of
> > > alternative prefixes and that is bad for everyone.
> > 
> > I think we'll have to disagree about what encouragement is, then.  IMHO the
> > option enables, but doesn't encourage.
> 
> "Hey, this software has a knob to enable the thing I was just thinking
> I should do.  Must be OK!"

Er, I meant to say (explicitly) that providing a feature to enable
users' bad ideas is certainly a form of encouragment.  At the very
least, it can be considered indirect encouragment.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: $reply_prefix

[Derek Martin]

On Fri, Jan 14, 2022 at 09:37:06AM +0100, Vincent Lefevre wrote:
> On 2022-01-13 19:41:12 -0800, Will Yardley wrote:
> > On Thu, Jan 13, 2022 at 09:42:27PM -0500, John Hawkinson wrote:
> > > Perhaps language from RFC5322 Sec. 3.6.5 should be imported into the
> > > documentation for $reply_prefix, but even that is oddly permissive
> > > ("When used in a reply, the field body MAY start with the string "Re:
> > > "") and it would be weird to change it to MUST, but I suppose wcould.
> >  
> > I looked this up as well -- since it's a "MAY", I would agree that the
> > change probably does _not_ technically violate the RFC.
> 
> The context is about the subject of a topic:
> 
>   The "Subject:" field is the most common and contains a short string
>   identifying the topic of the message.
> 
> So, the subject is not expected to change in a reply, otherwise the
> reply will be regarded as a different topic. What the RFC says is
> that there is an exception: "Re: " may be added before the contents
> of the "Subject:" field body of the original message. I suppose that
> the alternative is to leave the subject as is, without any prefix
> (note that the "In-Reply-To:" header is sufficient to indicate that
> this is a reply).

FWIW, I agree 100% with this interpretation as well.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: $reply_prefix

[Derek Martin]

On Thu, Jan 13, 2022 at 09:25:27PM -0800, Kevin J. McCarthy wrote:
> On Thu, Jan 13, 2022 at 11:57:37PM -0500, John Hawkinson wrote:
> > Kevin J. McCarthy  wrote on Thu, 13 Jan 2022
> > at 23:20:55 EST in :
> > 
> > > I've been told other prefixes are often used in some lists, and the 
> > > practice
> > > is getting more common.  Why not give users the option to adjust it, if 
> > > they
> > > deem it appropriate, for some lists?
> > 
> > The reason not to is that the knob encourages the proliferation of
> > alternative prefixes and that is bad for everyone.
> 
> I think we'll have to disagree about what encouragement is, then.  IMHO the
> option enables, but doesn't encourage.

"Hey, this software has a knob to enable the thing I was just thinking
I should do.  Must be OK!"

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: $reply_prefix

[Derek Martin]

On Fri, Jan 14, 2022 at 03:26:36AM +0100, Vincent Lefevre wrote:
> I strongly disagree with the addition of $reply_prefix
> (commit 9c1ce59874ce1c8e97d0c5bd71847596dafb1d50), as this is
> contrary to RFC 5322, and non-standard prefixes are annoying
> in practice and not necessarily recognized by other users.
> 
> Sure, a user could already change it manually (or automatically
> with things like wrappers), but he should not be encouraged to
> do so (in particular by making this automatic). End users are not
> necessarily aware of RFCs and technical issues behind some choices.

I wholeheartedly agree.  I'm pretty sure I already said that
somewhere... or at least meant to. :)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt on ubuntu 20

[Derek Martin]

On Fri, Jan 14, 2022 at 03:19:39AM +0100, Vincent Lefevre wrote:
> On 2022-01-13 15:36:04 -0600, Derek Martin wrote:
> > So...  The only difference between this and hdrdefault is that
> > hdrdefault's order does not matter.  This is a very minor difference,
> > and the distinction disappears so long as you provide the "." rule
> > first,
> 
> This would mean that it would not be possible to change the default
> header color from a running Mutt.

No it doesn't; use uncolor, then enter the rules in the correct order.

I'm well aware doing that would be dumb, but it IS possible.  It would
be far more sensible to just restart Mutt with the updated config,
which is what any reasonable person would do, and which is necessary
in any case to make the change permanent.  This is such an uncommon
case that considering it seems silly, but regardless, it's false.

> Currently, my header color settings do not depend on any order as they
> are mutually exclusive

That's possible, if your rules are very carefully crafted; but it's
not generally true as Mutt's regex rules are inherently ordered, and
if it is true in your case it is specific to your rules, not Mutt's
functionality generally.  But it's irrelevant; the config language not
having the precise semantics you personally prefer does not change the
fact that you STILL CAN achieve the same result without the default
keyword.

Redundant.

> > just as you must already do with send-hook and every other
> 
> I don't use send-hook (and other hooks) on "color header".

That's not what I'm saying.

# apply a default folder hook
folder-hook . set x=foo
# other folders
folder-hook work set x=bar
folder-hook ice-cream set x=baz
...

The . rule must come first.  There is no default folder-hook.  The
necessity of ordering has precedence in Mutt and is inherent in any
case.

And yes, ice cream definitely deserves its own mail folder. =8^)

I've had my fill of this pointless argument.  Especially pointless
since I'm not actually advocating removing it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt on ubuntu 20

[Derek Martin]

On Thu, Jan 13, 2022 at 06:22:26PM +0100, Vincent Lefevre wrote:
> On 2022-01-12 14:40:38 -0600, Derek Martin wrote:
> > FWIW (particularly with this fixed) I think hdrdefault is redundant,
> 
> I don't think it is. IMHO, hdrdefault means the color when no other
> rules match. If you use "color header ... .", it would override
> other rules earlier in the list.

I'll first point out that this is entirely a semantic argument, not a
technical one. I've already said I don't think this should have any
bearing on whether this get fixed, for whatever that's worth.

"Redundant" has a number of meanings, the relevant one here being "(of
words or data) able to be omitted without loss of meaning or
function."[1]

So...  The only difference between this and hdrdefault is that
hdrdefault's order does not matter.  This is a very minor difference,
and the distinction disappears so long as you provide the "." rule
first, just as you must already do with send-hook and every other
thing mutt provides with similar semantics, which don't have explicit
default keywords.  Users familiar with other similar Mutt features
should and would expect that, and in any case it should be likewise
documented.

Thus hdrdefault can be omitted by ensuring that a "." rule appear
first in the list of rules, without loss of function (assuming of
course that this were fixed, as I previously said): Therefore it is
redundant, by definition.  If you want to argue that redundant means
something different, please feel free to contact the fine folks at
Oxford.  In any case this is precisely the meaning I intended when I
wrote it.

-=-=-

[1] Google's dictionary, provided by Oxford Languages.
https://www.google.com/search?q=redundant=3ZfgYceaNIOC-QaBnZfQCw=0ahUKEwiH_bOq1K_1AhUDQd4KHYHOBboQ4dUDCA4=5=redundant_lcp=Cgdnd3Mtd2l6EAMyBwgAELEDEEMyBwgAELEDEEMyCAgAEIAEELEDMggIABCABBCxAzIFCAAQgAQyBQgAEIAEMgUIABCABDIFCAAQgAQyBQgAEIAEMgUIABCABDoHCAAQRxCwAzoHCAAQsAMQQzoICAAQ5AIQsAM6EAguEMcBENEDEMgDELADEEM6CgguEMgDELADEEM6CAguELEDEIMBOgUILhCABDoICC4QgAQQsQM6EQguEIAEELEDEIMBEMcBENEDOg4ILhCABBCxAxDHARCjAjoFCC4QsQM6BQgAEJECOgQIABBDOgUILhCRAjoECC4QQzoRCC4QsQMQgwEQxwEQ0QMQkQI6CAguELEDEJECOgoILhDHARCvARBDOgcIABDJAxBDSgQIQRgASgQIRhgBUPQFWPYSYP8TaAFwAngAgAFTiAHbBJIBATmYAQCgAQHIARLAAQE=gws-wiz

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt on ubuntu 20

[Derek Martin]

On Tue, Jan 11, 2022 at 05:53:45PM -0800, Kevin J. McCarthy wrote:
> On Tue, Jan 11, 2022 at 03:32:09PM -0600, Derek Martin wrote:
> > OK but do you agree that behavior is wrong? :) It's one header,
> > regardless of the wrapping...
> 
> I'll have to think more closely about it.  I do agree that the matching on
> the folded line overriding the previous 'color header' match is confusing.
> I don't believe the fix is hard, either.

FWIW/TLDR, I think as a matter of principle it should be fixed, and I
doubt there can be any compelling arguments (beyond not being willing
to volunteer the time to do it, which is a good one) that it
shouldn't.  I won't be heartbroken if it doesn't get fixed since I
have config that works, but I do think not fixing the inconsistency
makes Mutt suck just a little bit more...

Now, the details:

> However, it's not just about wrong vs. right

I do agree--wrong is wrong, but there are two other questions to
answer. There's the question of, "Does this provide useful
functionality for some real-world use case?"  I think the answer is
that it does not. As you yourself said, the behavior is confusing,
which does not lend itself to real-world use.  As such the
behavior is, I think, clearly broken.  

>  but also about precedence of the behavior, and I believe this one
>  goes back to 1.5.21.  If I make the change, I may then have
>  long-time users reporting a "genuine" regression.

To me this argument is a much weaker one--wrong is wrong, and
depending on a bug is a bug. :)  I don't think this is one of the two
questions... not precisely anyway (see below).

Also, this DID work correctly (i.e. as I believe it should work), in
for example 1.4.2.3 (I did actually check).  So this IS a genuine
regression... just one that no one has apparently noticed (or rather,
reported) for a very, very long time.  I didn't do the binary search
to see where it stopped, I'll trust your suggestion that it was
1.5.21. :)

[FWIW this IS something I have noticed on occasion that has been
bugging me for a long while... but it didn't happen enough to distract
me enough from what I was doing to actually look into it.  Lately it
has though, and I understand exactly why:  As my professional role and
responsibilities have expanded, so too have the number of mail threads
I've been on with larger recipient lists.  Otherwise, the messages I
send and receive very typically have at most one or two people, and/or
a mailing list.  Not enough to warrant wrapping of the recipient
headers, and therefore triggering this bug.]

The second question is the matter of consequences:  If you correct the
(I think) clearly broken behavior, what is the impact on anyone
affected?  Well, in the worst case scenario, where someone actually is
depending on this behavior (which I think is pretty unlikely, given
much of the above), they'll just have to fix probably one or two color
commands in their muttrc, or else some headers may be the wrong color.
Admittedly a minor annoyance--not enough, I think, to offset the
behavior being correct.   In probably the vast majority of cases, it
will require no action, either because the user isn't using this
feature at all, or because they actually intended it to work as it did
in the 1.4 era, and just never noticed or cared enough to bother to
report that it didn't, like me.

Now, for the bonus round:

FWIW (particularly with this fixed) I think hdrdefault is redundant, and
could actually be removed, though that would force some users to
change their muttrc (so I'm not suggesting that, just pointing it
out).  However, assuming it's implemented as I imagine (I still
haven't looked at the code, FWIW) it does reduce the regex comparisons
required by 1, providing a slight optimization.  Although, arguably,
it would have exactly the same effect if the regex evaluation was
optimized according to this python-like pseudo code:

rule_applied = None
foreach rule in rules:
# Optimize the "match all" rule...
# Prolly document that "." is shorthand for "match everything
# by default" and don't try to be too clever about what other
# regexes to optimize...
if rule.regex == ".":
# everything matches "." -- don't bother checking
rule_applied = rule
else:
if rule.regex matches text:
rule_applied = rule

# all rules checked--apply the last match.
if rule_applied:
apply rule_applied

In practice, I doubt this matters enough for anyone to notice the
optimization, but possibly with lots of rules on slow hardware...
Nah, still probably not. :)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt on ubuntu 20

[Derek Martin]

On Tue, Jan 11, 2022 at 11:48:38AM -0800, Kevin J. McCarthy wrote:
> On Tue, Jan 11, 2022 at 12:48:03PM -0600, Derek Martin wrote:
> > On Mon, Jan 10, 2022 at 05:27:32PM -0800, Kevin J. McCarthy wrote:
> > > Can you provide a minimal reproduce of the problem?
> > 
> > I sure can:
> > 
> > set spoolfile = Path/to/coninued/header/message/folder
> > color normal default default
> > color header brightgreen default .
> > color header brightmagenta default ^(subject|from|to|cc|date)
> > 
> > With these, the specified headers will all be magenta.  That is,
> > unless they are continued--then they will be green.
> 
> I can reproduce this with versions 1.10.1 and 1.11.4, and so I don't believe
> this is a regression.

Turns out I have 1.9.4 lying around and I can repro it there as well.

> When the header wraps, the second color line above will match the wrapped
> line, and will then override the third color lines' match of the
> first header line.

OK but do you agree that behavior is wrong? :)  It's one header,
regardless of the wrapping...

> I think instead you probably want:
> 
> color normal default default
> color hdrdefault brightgreen default
> color header brightmagenta default ^(subject|from|to|cc|date)

That does seem to work... I wasn't familiar with that keyword.  Even
after 25 years, I don't know all the keywords.  Just supports my
theory that Mutt has too many, IMO. =8^)

At any rate, given my way has parity with configuring hooks and such,
and is (IMO) the intuitive behavior, I think the two versions of
config ought to work identically.  YMMV.

> > On a separate note, I tried to compile 2.1.5 with the sidebar enabled,
> > and I got this mess (which I made no effort to investigate, but I can
> > later, theoretically, if you like), just FYI:
> 
> I can't reproduce this from a fresh unpacked tarball.
[...]
> Try a make clean; make and see if that fixes the problem?

It did!  I should have tried that outright, but... too many pots on
the stove.


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt on ubuntu 20

[Derek Martin]

On Mon, Jan 10, 2022 at 04:45:04PM -0800, Dan Fandrich wrote:
> On Mon, Jan 10, 2022 at 06:27:04PM -0600, Derek Martin wrote:
> > 1. Does anyone have any idea why ubuntu is shipping such an ancient
> >version of Mutt? ;-)
> 
> 1.13.2 was the most recent release of mutt at the beginning of 2020 when 
> Ubuntu
> 20.04 went into testing

So it was... I imagined that version was considerably older.  A
testament to Kevin's progress. :)


On Mon, Jan 10, 2022 at 05:27:32PM -0800, Kevin J. McCarthy wrote:
> On Mon, Jan 10, 2022 at 06:27:04PM -0600, Derek Martin wrote:
> > 2. It appears that this version of Mutt does not properly honor color
> >   commands in headers that are continued in multiple lines.  Is this
> >   a bug that was found and fixed in a future version?
> 
> Derek, I can't recall a recent fix pertaining to that, but perhaps I've
> forgotten.

I went back after I finished what I was working on, and compiled the
latest mutt.  I can confirm the bug is still present.

> Can you provide a minimal reproduce of the problem?

I sure can:

set spoolfile = Path/to/coninued/header/message/folder
color normal default default
color header brightgreen default .
color header brightmagenta default ^(subject|from|to|cc|date)

With these, the specified headers will all be magenta.  That is,
unless they are continued--then they will be green.

Possibly useful to know:  I'm using gnome-terminal, with TERM forced
to xterm (because I don't actually have a use for the 256-color
behavior), and the option to use bright colors for bold (so you can
actually use the top 8 colors) is checked.  Although, I just tried it
with my ancient xterm (actual xterm) config and it behaves the same
way there as well.  Point is, I'm not using a 256-color config.

> You don't have $header_color_partial set, do you?

No.

-=-=-=-

On a separate note, I tried to compile 2.1.5 with the sidebar enabled,
and I got this mess (which I made no effort to investigate, but I can
later, theoretically, if you like), just FYI:

cc -DPKGDATADIR=\"/usr/local/share/mutt\" -DSYSCONFDIR=\"/usr/local/etc\" 
-DBINDIR=\"/usr/local/bin\" -DMUTTLOCALEDIR=\"/usr/local/share/locale\" 
-DHAVE_CONFIG_H=1 -I.  -I. -I.  -Wall -pedantic -Wno-long-long -g -O2 -MT 
curs_main.o -MD -MP -MF .deps/curs_main.Tpo -c -o curs_main.o curs_main.c
curs_main.c: In function ‘mutt_index_menu’:
curs_main.c:1293:12: error: ‘OP_SIDEBAR_OPEN’ undeclared (first use in this 
function)
 1293 |   case OP_SIDEBAR_OPEN:
  |^~~
curs_main.c:1293:12: note: each undeclared identifier is reported only once for 
each function it appears in
curs_main.c:2617:12: error: ‘OP_SIDEBAR_FIRST’ undeclared (first use in this 
function)
 2617 |   case OP_SIDEBAR_FIRST:
  |^~~~
curs_main.c:2618:12: error: ‘OP_SIDEBAR_LAST’ undeclared (first use in this 
function)
 2618 |   case OP_SIDEBAR_LAST:
  |^~~
curs_main.c:2619:12: error: ‘OP_SIDEBAR_NEXT’ undeclared (first use in this 
function); did you mean ‘OP_SEARCH_NEXT’?
 2619 |   case OP_SIDEBAR_NEXT:
  |^~~
  |OP_SEARCH_NEXT
curs_main.c:2620:12: error: ‘OP_SIDEBAR_NEXT_NEW’ undeclared (first use in this 
function); did you mean ‘OP_MAIN_NEXT_NEW’?
 2620 |   case OP_SIDEBAR_NEXT_NEW:
  |^~~
  |OP_MAIN_NEXT_NEW
curs_main.c:2621:12: error: ‘OP_SIDEBAR_PAGE_DOWN’ undeclared (first use in 
this function)
 2621 |   case OP_SIDEBAR_PAGE_DOWN:
  |^~~~
curs_main.c:2622:12: error: ‘OP_SIDEBAR_PAGE_UP’ undeclared (first use in this 
function)
 2622 |   case OP_SIDEBAR_PAGE_UP:
  |^~
curs_main.c:2623:12: error: ‘OP_SIDEBAR_PREV’ undeclared (first use in this 
function)
 2623 |   case OP_SIDEBAR_PREV:
  |^~~
curs_main.c:2624:12: error: ‘OP_SIDEBAR_PREV_NEW’ undeclared (first use in this 
function); did you mean ‘OP_MAIN_PREV_NEW’?
 2624 |   case OP_SIDEBAR_PREV_NEW:
  |^~~
  |OP_MAIN_PREV_NEW
curs_main.c:2628:12: error: ‘OP_SIDEBAR_TOGGLE_VISIBLE’ undeclared (first use 
in this function)
 2628 |   case OP_SIDEBAR_TOGGLE_VISIBLE:
  |^
make[2]: *** [Makefile:908: curs_main.o] Error 1
make[2]: Leaving directory '/home/demartin/Downloads/mutt-2.1.5'
make[1]: *** [Makefile:928: all-recursive] Error 1
make[1]: Leaving directory '/home/demartin/Downloads/mutt-2.1.5'
make: *** [Makefile:614: all] Error 2


My configure line:

./configure  --enable-sidebar --disable-external-dotlock --enable-hcache

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Option to clear the screen on quit

[Derek Martin]

On Mon, Jan 10, 2022 at 06:21:58PM -0600, Derek Martin wrote:
> > I much prefer:
> > #!/bin/sh
> > absent some special reason. /bin/sh exists on all POSIX systems, and 
> > bash need not (and if it does, it is not always in /bin).
> 
> I agree wholeheartedly, and FWIW invoking as bash changes the behavior

Sorry for waking up this thread, I had this message postponed from 2
months ago and meant to kill it, sent it instead. ^_^;

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Mutt on ubuntu 20

[Derek Martin]

I recently was given a laptop by my employer, running Ubuntu 20.  I
have some disincentive (which I may ignore, given sufficient reason)
to run what they provide.  It seems to ship with mutt-1.13.2.  So, now
I have TWO questions...

1. Does anyone have any idea why ubuntu is shipping such an ancient
   version of Mutt? ;-)

2. It appears that this version of Mutt does not properly honor color
   commands in headers that are continued in multiple lines.  Is this
   a bug that was found and fixed in a future version? (I can
   obviously download the source and try it but that takes some time I
   don't really have right at the moment, so if someone knows the
   answer, all the better).

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Option to clear the screen on quit

[Derek Martin]

On Mon, Nov 01, 2021 at 11:47:40AM +1100, Cameron Simpson wrote:
> On 17Oct2021 15:04, Oskari Pirhonen  wrote:
> >#! /bin/bash
> 
> I much prefer:
> #!/bin/sh
> absent some special reason. /bin/sh exists on all POSIX systems, and 
> bash need not (and if it does, it is not always in /bin).

I agree wholeheartedly, and FWIW invoking as bash changes the behavior
of scripts in subtle but possibly undesireable ways.  Probably won't
affect most people but if you're a "power user" and use certain
features of the shell (such as for example setting ENV) it can...

The only downside is these days on many systems /bin/sh is dash which
annoyingly behaves differently than bash, again in mostly subtle ways,
but one common one you might hit is options for the echo command.

> >eval $(which mutt) "$@" && clear
> 
> Others have wondered about the "which". I wonder about the "eval", which 
> is just asking for attack (or unhappy accidents). Please try not to use 
> it.

Agree on this as well.  Someone commented that which is deprecated on
debian unstable--this seems like a pointless and stupid change.  Every
Unix system I can think of has had a which command going back maybe 40
years... deprecating it has the potential to break a lot of things for
as far as I can see no practical reason whatsoever.

This is not progress.

On Mon, Nov 01, 2021 at 11:44:25AM +1100, Cameron Simpson wrote:
> On 22Oct2021 10:30, Kevin J. McCarthy  wrote:
> [...]
> >After 25 years of this behavior, is clearing the alternate screen 
> >really now a security or privacy issue; against an attacker who 
> >evidently has access to your terminal?

Hopefully we've established that it isn't.

> That said, and ignoring the alternate screen, I've noted with annoyance 
> that "clear" in iterm doesn't erase.

I don't think that's actually true--at least it isn't for me.  It
absolutely does clear the terminal...  But what it does not do is
clear the scroll-back buffer.  I would prefer it did, also.  It may
have a setting for that which I haven't discovered.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Enabling $rfc2047_parameters by default

[Derek Martin]

On Sat, Jan 08, 2022 at 02:46:48PM -0800, Kevin J. McCarthy wrote:
> I'm thinking about enabling $rfc2047_parameters by default, and would like
> to hear any counter-arguments against this.

I believe the original argument against was that doing so violates the
RFCs, and therefore potentially obscures a header that actually wanted
that text to appear in the header in conformance with the spec.

However--and my memory on this is as vague as ever--wasn't there an
update to the RFCs that expressly allowed it in headers for which it
wasn't previously allowed?

One certainly might raise the question of why it was originally
excluded from the spec...  There was probably a reason, but I don't
know it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Use From-address fqdn for Message-ID Generation

[Derek Martin]

On Tue, Nov 23, 2021 at 04:40:28PM -0600, Aaron Poffenberger wrote:
> As for making the fqdn of the From-address available during Message-ID
> generation, I see value in it regardless. But it's not my software. I
> don't get to decide. I'll leave it as it always was: a suggestion.

At any rate, this is simply wrong, and Mutt should neither support nor
allow it.  RFC 2822 says:

   The "Message-ID:" field provides a unique message identifier that
   refers to a particular version of a particular message.  The
   uniqueness of the message identifier is guaranteed by the host that
   generates it (see below).
   ...
   The message identifier (msg-id) itself MUST be a globally unique
   identifier for a message.  The generator of the message identifier
   MUST guarantee that the msg-id is unique.

If you allow the user to muck with the host portion (techniqally
dot-atom-text portion) of the message ID you make it impossible to
guarantee that the message ID so generated is globally unique.  In
fact it would generally be better for Mutt to not set the message ID
*at all*, and allow the SMTP server it submits the message to, to do
that, which would guarantee uniquness so long as the SMTP server is
sane and reasonably well-behaved... but which is not Mutt's concern at
any rate.

As it stands currently, if Mutt is using the same format as the user's
domain's mail server to generate message IDs, it's possible for Mutt
and the mail server to generate the same ID for different messages. I
still maintain that allowing the user control over the message ID is a
bad idea.  In the best case, duplicate IDs make threading broken; in
the worst case it causes messages to be lost.

From https://en.wikipedia.org/wiki/Message-ID:

No two different messages must ever have the same Message-ID. If
two messages have the same Message-ID, they are assumed to be the
same and one version is discarded. This can cause issues if tools
mangle the IDs created by other tools. Such a problem has been
reported with Google MTAs mangling Message-IDs created by Outlook,
making it difficult to reference other messages and breaking
threading.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Use From-address fqdn for Message-ID Generation

[Derek Martin]

On Tue, Nov 23, 2021 at 05:52:00PM -0600, Aaron Poffenberger wrote:
> On 2021-11-23 17:28 -0600, Derek Martin wrote:
> > > My mail domain has had correct reverse DNS entries for ages.  As soon
> > > as I received the error I verified it from multiple hosts to be sure.
> > 
> > Well, the IP you gave for it does not resolve, as I already showed
> > you--not on my laptop at home, not on my work deskop, and not on my
> > hosted domain.  So unless that IP was wrong/irrelevant, that doesn't
> > seem to be true.  I can't speak to whatever testing you did but there
> > are potentially a number of reasons why you might get the answer you
> > expect while the rest of the internet doesn't.
> 
> That IP address purposely doesn't resolve. It's in the range reserved
> for documentation to parallel the uses of example.com in the
> narrative.[0][1]

My mistake, I was not familiar with reserved blocks other than
RFC1918.  That sort of sysadmin stuff hasn't been in my purview for
about 15 years now...

> Could also be I'm not clueless and made a real, informed effort.

One need not be clueless to not know a critical piece of
information. =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Use From-address fqdn for Message-ID Generation

[Derek Martin]

On Mon, Nov 22, 2021 at 11:22:33AM -0600, Aaron Poffenberger wrote:
> On 2021-11-22 16:48 +, Claus Assmann  wrote:
> > I'm not sure the problem is really related to "fqdn of the Message-ID",
> > could you give a real example:
> > sending from the same host using the same addresses but two different
> > Message-ID - only one of which fulfills the requirement you stated.

Clauss et al. are correct.  The SMTP server does not look at the
contents (i.e. the message headers) to make this decision; it is based
solely on the message envelope presented in the SMTP conversation.
That conversation can be and often is entirely different from what's
in the headers, and you generally don't ever get to see it.  The
server has given you an honest accounting of the error..

> sbcglobal.net will fail delivery of the message with the following error:
> 
> @*sbcglobal.net*: 550 5.7.1 Connections not accepted from
> servers without a valid sender domain.alph758 Fix reverse DNS for 198.51.100.1

It's just told you what the problem is, and you've misunderstood or
ignored the explanation, and come up with a solution that just happens
to work by coincidence, because it *also* effectively changes the
piece of data that actually matters...

Most likely the failed SMTP conversation (in its entirety) looks
something like this:

-=-=-=-=-
EHLO host.example.com
550 5.71 Connections not accepted from servers without a valid sender 
domain.alph758 Fix reverse DNS for 198.51.100.1
-=-=-=-=-

The receiving SMTP server most likely never gets past this--never
gives your laptop a chance to send the message data (including the
completely irrelevant message ID and the rest of the headers), because
it first tries to look up host.example.com, which as you've said has
no A record, so that fails.  The IP address 198.51.100.1 also does not
have a PTR record:

$ host 198.51.100.1
Host 1.100.51.198.in-addr.arpa. not found: 3(NXDOMAIN)

Normally the server will check the forward and reverse DNS lookups
match (there's some minor variation and configurable behavior around
this check from server to server, including the possibility to enable
or disable it entirely), and reject the connection if there's a
problem. I'm 99.9% sure that is what has happened here.

The usual (and most correct) way around this problem is to set the
outgoing mail relay on the machine from which you're sending mail to
be the mail server responsible for your domain, assuming it is
propertly configured to accept mail to be relayed from your machine...
If it's your ISP's server or similar, there may be hoops you need to
jump through to configure authentication so that it will do that.

Bottom line:  This is NOT a Mutt problem.  It's a misconfigured
MTA/host problem.  Without a MUCH better justification, this patch
should be rejected: It only allows the user to configure their message
ID improperly.  But then again, I made that same argument for the
entire message ID format feature, and pointed out that if you let
people do that, they will...

> I know this is related to Message-ID...

The most confounding part of being human is you can be absolutely
certain that something is true, and still be wrong... =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Option to clear the screen on quit

[Derek Martin]

On Tue, Nov 02, 2021 at 07:10:24PM +0100, Petr Pisar wrote:
> V Mon, Nov 01, 2021 at 11:44:25AM +1100, Cameron Simpson napsal(a):
> For that purposes I added ^[3J sequence into Linux 3.0 which erases not only
> whole display but also a scroll-back buffer of the terminal. (Though Linux
> 4.something removed scrollback buffer completely, you cannot Shift+PgUp
> anymore.)
> 
> Maybe it's time to send a feature request to a terminal emulator of your
> choice to implement it. (I remember xterm's maintainer was not against.)
> clear of ncurses emmits it:
> 
> $ clear | hexdump -C
>   1b 5b 48 1b 5b 32 4a 1b  5b 33 4a |.[H.[2J.[3J|
> 000b
> 
> Naturaly it should also extend to altearnative screens. As the purpose of the
> sequence is to erase all data for security reasons.

No that's not "natural."  The whole point of having alternative
screens is to retain the data on one while you do work in the other.
If you want the alternate screen cleared, clear it.  If worse comes to
worst, close the terminal.  A command-line option to clear might be
nice to make this easy for the user, but it should not be the default
behavior.  Then if it really is what you want, all the time, you can
just make a shell alias:

alias clear="clear --clear-alternate-screen-also"


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Option to clear the screen on quit

[Derek Martin]

On Tue, Nov 02, 2021 at 12:17:17PM +0100, Vincent Lefevre wrote:
> On 2021-10-29 14:19:29 -0500, Derek Martin wrote:
> > On Wed, Oct 27, 2021 at 05:10:35PM +0200, Vincent Lefevre wrote:
> > > I was wondering whether this could occur when switching to the
> > > alternate screen. But it seems that this is not the case, at least
> > > not with Xterm's logging feature.
> > 
> > Right, that wouldn't make sense... the I/O is not re-delivered to the
> > terminal (and I assume the contents of the alternate screen are just
> > sitting in a buffer, which is simply re-displayed).  Otherwise
> > switching back and forth between screens would cause the data to be
> > sent to the printer, again and again...  That does not seem like it
> > would ever be desireable behavior.
> 
> Well, for lots of bugs, the behavior doesn't make sense. That's why
> they are bugs. Thus it may be a good idea not to leave private data
> behind, in case there would be such a bug.
> 
> Similarly, there is (was?) this old Xorg/nouveau bug that may reveal
> parts of the user's desktop after reboot (which is problematic on a
> shared computer). Displaying old video memory (not erased at logout
> or at shutdown) doesn't make sense, but this happened.

Yeah but those are actual bugs, where leaving data behind on the
user's terminal's alternate screen IS NOT.  It's designed to work that
way, and the design is correct... it's largely the reason alternate
screens exist.  Re-dumping the data to your printer would be a bug in
your terminal.

If your terminal has a bug where that's a problem, any related
security issue is in the terminal, NOT in Mutt.  I think you can make
a pretty good case that Mutt should work around stupidity in broken
e-mail clients (like all the various brain damage in Outlook), since
it is directly related to, and directly affects, Mutt's primary
function.  But I don't think you can make a good case that Mutt should
try to work around every possible bug in every possible software
component of your system, since that is a practical impossibility.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Option to clear the screen on quit

[Derek Martin]

On Wed, Oct 27, 2021 at 05:10:35PM +0200, Vincent Lefevre wrote:
> On 2021-10-25 14:44:32 -0500, Derek Martin wrote:
> > There is an ANSI escape sequence to tee data to your printer, sure...
> > but it can not be used retroactively copy data that is on your
> > terminal to the printer.  It just copies data that is currently being
> > displayed (i.e. since the sequence was emitted) to the printer.
> 
> I was wondering whether this could occur when switching to the
> alternate screen. But it seems that this is not the case, at least
> not with Xterm's logging feature.

Right, that wouldn't make sense... the I/O is not re-delivered to the
terminal (and I assume the contents of the alternate screen are just
sitting in a buffer, which is simply re-displayed).  Otherwise
switching back and forth between screens would cause the data to be
sent to the printer, again and again...  That does not seem like it
would ever be desireable behavior.

> So I assume that as long as the user doesn't use a virtual terminal
> inside the real terminal, things are safe. Users of virtual terminals
> (GNU Screen, etc.) should be careful, as older data are sent back to
> the real terminal when switching a window, for instance.

I hadn't considered this, but one would hope that maintainers of such
software would not buffer the printer sequence so that it is only
issued on initial display.  In any case...

> However, in case of any issue, the real solution should be to ensure
> that the printing feature is disabled.

I would not disagree. :)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Option to clear the screen on quit

[Derek Martin]

On Sat, Oct 23, 2021 at 02:04:08PM +0200, Vincent Lefevre wrote:
> On 2021-10-22 10:30:43 -0700, Kevin J. McCarthy wrote:
> > On Fri, Oct 22, 2021 at 12:51:04PM +0200, Vincent Lefevre wrote:
> > > Following my remark about the privacy reason, I think that the patch
> > > would be useful to make sure that data are not silently left on the
> > > alternate screen (which is no longer visible after quitting Mutt,
> > > but can be retrieved at least in xterm with its menus, and possibly
> > > via escape sequences I think).
> > 
> > I don't feel _strongly_ against the patch, but absent a real threat model
> > I'm still not convinced it belongs inside Mutt.
> > 
> > After 25 years of this behavior, is clearing the alternate screen really now
> > a security or privacy issue; against an attacker who evidently has access to
> > your terminal?
> 
> There might still be a possibility to have the contents printed via
> escape sequences, e.g. with some shell command. For instance, what
> happened to me in the past:
> 
>   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=301989
> 
> (FYI, the data were sent on a *shared* printer in my lab).

There is an ANSI escape sequence to tee data to your printer, sure...
but it can not be used retroactively copy data that is on your
terminal to the printer.  It just copies data that is currently being
displayed (i.e. since the sequence was emitted) to the printer.

I think the notion that this is a privacy/security concern is nuts.
In the unlikely scenario you left something sensitive on your terminal
window when you exited mutt, andf you're *really* that paranoid, clear
the screen from your terminal's menu before leaving it, or close your
terminal window.  If you leave your computer unattended and unlocked,
giving random passers-by access to your terminals, THAT, not this, is
the security issue.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: strfcpy in dotlock.c

[Derek Martin]

On Fri, Jul 23, 2021 at 03:30:01AM +0200, Vincent Lefevre wrote:
> On 2021-07-22 09:11:56 -0700, Kevin J. McCarthy wrote:
> > What do you think of using memccpy in the strfcpy macro,
> 
> IMHO, it is better than strncpy, which unnecessarily pads with
> null bytes.
> 
> > if it's available, as Ian suggested?
> 
> The Linux memccpy(3) man page says:
> 
> CONFORMING TO
>POSIX.1-2001, POSIX.1-2008, SVr4, 4.3BSD.
> 
> So, for Mutt, it should be available everywhere.

This is probably the best implementation for mutt, and makes sense
where e.g. the string is an index format string etc., where truncating
the string is normal and expected.

If mutt is going to use strfcpy elsewhere, where truncating the string
silently might be dangerous, e.g. potentially when programmatically
determining a filename, or similar, you should probably have a more
secure implementation that doesn't allow you to silently truncate the
string.  Something like this:

int scopy(char *dest, char *src, size_t len)
{
int rc = snprintf(dest, len, "%s", src);
if (rc >= (int)len) return -1;
else return rc;
}

It may make sense to have both...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Mon, Jan 11, 2021 at 08:22:08AM +, Eric Wong wrote:
> Derek Martin  wrote:
> Well, mistakes happen...  How we deal with them moving forward
> is important, though.

I certainly agree with that.

> Fwiw, I concur the PID in Message-IDs was a bad idea since it
> does unnecessarily leak information about system usage.  I'm no
> security expert

By your own admission, this is a naive (uninformed) opinion.  No one
has been able to demostrate that the PID is in any way sensitive in
this context, and no one ever will, because it isn't.

> though I seem to remember kernel patches that would randomize PID
> allocation in Linux.

The only way a PID could be sensitive is if it is used to generate a
piece of information that is sensitive in a security context, e.g. a
cookie, or a temporary file name.  This is the purpose for the PID
randomization patch.  The bad news is, if it WERE sensitve, you'd be
pretty screwed, because most systems only allow 32k of them, which
would make it extremely easy to brute-force attack.

As it turns out, this is very likely an extremely bad solution to
those types of problems:

  
https://security.stackexchange.com/questions/88692/do-randomized-pids-bring-more-security

But it doesn't matter--this is just so much a non-issue for message
IDs.  We shouldn't be mucking with this.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Jan 09, 2021 at 10:54:28PM +, Eric Wong wrote:
> Hi Remco,
> 
> So I'm looking at 9da4e6e11e7037668d0ca7e8f5d6773d26e379ac
> (I noticed this in mutt 2.0.2 on FreeBSD)
> 
> This is a bad change

You should have just stopped there.  This is a bad change, as I argued
when it was originally discussed.  Given that there was neither
agreement about how to change it, even amongst those who thought it
should change, nor any concrete example of harm caused by the original
format, it should never have been changed.  

This thread is a good example of why you don't arbtrarily change
things that have worked the same way for 20 years without a very good
reason:  You have NO IDEA what the change will break or how many
people will be affected by the breakage.  For all of the above
reasons, the new patch should not be accepted, and the original change
should be REVERTED.

FWIW, the message ID should NOT be URL-encoded.  That is the job of
the web interface for the archives/whatever.  Using such an encoding
makes it harder for the humans to read and compare message IDs, but
programs have no trouble doing such conversions as needed.  The only
time it should be URL-encoded is when it is in a URL. Mutt doesn't
generate those.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

On Thu, Nov 12, 2020 at 04:03:28PM +, Ian Collier wrote:
> On Thu, Nov 12, 2020 at 09:22:45AM -0600, Derek Martin wrote:
> > One point that is not clear to me is, the sigaction manpage explicitly
> > calls out execve, rather than the whole exec family.  It seems to
> > suggest that execvp (which is what mutt is using to spawn the sendmail
> > process) doesn't do that.  It could be a case of the man page being
> > incorrectly overly specific, or just out of date... 
> 
> sigaction and execve are system calls in section 2 of the manual,
> whereas execvp et al are in section 3 and are library functions - in
> fact the man page says in the first paragraph that they are front-ends
> for execve(2).

I knew that... once. ;-)  I'm getting old, and most of the time these
esoteric details don't matter, so they're easy to forget if you
haven't bumped your head on them recently.

That explains it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

On Thu, Nov 12, 2020 at 12:18:07PM +0100, Oswald Buddenhagen wrote:
> On Thu, Nov 12, 2020 at 10:31:59AM +0100, Matthias Apitz wrote:
> > El día miércoles, noviembre 11, 2020 a las 02:34:57p. m. -0600, Derek
> > Martin escribió:
> > > > > It *is* supported by mutt using the following trick: one of the
> > > arguments
> > > > to mutt is:
> > > > >  | mutt -d4 -e "set sendmail=\"cat --\"" ... | sendmail -t
> > > 
> > > Having to use a hack rather means that it is NOT supported--only that
> > > doing arbitrary other unsupported things with hacks is supported. =8^)
> > > You should not think of Mutt as supporting this--it does not.
> > 
> > This is not a hack. The Mutt variable $sendmail is meant to specify any
> > process as MTA. Mutt is piping its output into the specified UNIX proc
> > and adds as arguments mail address(es) which we ignore in our case,
> > because the mail addr is specified in the mail itself as To: header.
> 
> > I can not see that using cat(1) as MTA and writing the message to STDOUT
> > is some kind of (illegal) hack.

No one said anything about it being illegal.  Mutt encourages hacks.
Mutt doesn't DIRECTLY support using sendmail this way.  You had to
write glue to make it work, even if that glue was just a cat command
line--which it wasn't, you actually had to write an external program
to do what you're doing.  Either way, that makes it a hack, by
definition.  But that doesn't make what you're doing "supported"--it
only makes it POSSIBLE.  Caveat emptor.

> that seems to be a common theme in your thinking. ;-) just because
> something is possible and not explicitly forbidden doesn't mean it's
> a good idea.  with this hack you have no error checking at all, and
> unexpected output on stdout would mess up things totally.

Agree, although I'm not sure about that last bit...  Mutt closes all
file descriptors (except stdin) before executing sendmail, some of the
time.  I didn't trace out exactly when.  When it does, that should be
impossible.  When it doesn't... I don't think we have enough info
about what/where stdout is.  It's possible that it's a pipe to one of
the other processes in the process pipeline.  If nothing is reading
it, the script may be blocked trying to write to stdout.  Or some
such.  But there's a fair likelihood given how it's running that it's
already /dev/null.

> > I'm already debugging our own application server which starts the shell
> > script creating all this process chain with system("script") and I
> > *think* this has set SIGCHLD to SIG_IGN
> > 
> there is no excuse whatsoever for spawning a child with suppressed sigchld
> (same for *most* other signals, for that matter). if this is indeed what is
> happening, you have your root cause right there (and my assessment from the
> other mail was in fact incorrect).

Agree.

> > (with good reasons,
> > 
> nonsense.

Agree.

> > because all the error handling should be done within "script" and not by
> > the application server, which has to serve Java written GUI frontends).
> > 
> that might be true, but is no excuse for ignoring errors that still come
> through, and even more for actively interfering with the child's execution
> environment.

Agree.

So I got this pretty wrong too, and admittedly didn't spend enough
time trying to understand some of the pieces.

One point that is not clear to me is, the sigaction manpage explicitly
calls out execve, rather than the whole exec family.  It seems to
suggest that execvp (which is what mutt is using to spawn the sendmail
process) doesn't do that.  It could be a case of the man page being
incorrectly overly specific, or just out of date...  Or it could be
that we still haven't nailed down the exact cause of Matthias'
problem.

Regardless, normally Mutt calls mutt_signal_init() which explicitly
sets the disposition of SIGCHLD, avoiding the above behavior.  However
run in a pipe as Matthias is doing here, it does not do that.  While I
think it's fair to say no developer ever anticipated this happening
for good reason, perhaps it makes sense for Mutt to unconditionally
reset the disposition of ALL signals to default before it does much of
anything else, to keep people from abusing it this way. But better
yet, just Don't Do That™.

I believe the rest of what I said about Mutt's handling of SIGCHLD,
its mishandling of waitpid(), and the negative consequences of the
double fork remain true, and are things that probably should be fixed.
However it's a fair amount of work, for a relatively small payoff of
correctness (disambiguated error conditions, but error conditions
nonethess), which the overwhelming majority of users will probably
never notice, and that's somewhat likely to intr

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

On Wed, Nov 11, 2020 at 08:50:27PM -0600, Derek Martin wrote:
> On Wed, Nov 11, 2020 at 02:34:57PM -0600, Derek Martin wrote:
> background.c:
>   background_edit_landing_page() has this code:
> 
> while (!done)
> {
>   wait_rc = waitpid (bg_pid, NULL, WNOHANG);
>   if ((wait_rc > 0) ||
>   ((wait_rc < 0) && (errno == ECHILD)))
>   {
> rc = 0;
> break;
>   }
> ...
> 
>   Checking for ECHILD is unnecessary if children are handled properly.

Ugh.  FWIW this also may not be quite right...  I originally had some
additional text intended to clarify that, but it got chopped as I
edited because the message was already so ridiculously long.  This may
well be right, if you have some expectation that the process SHOULD
still be running...  I didn't read enough of the background editing
code to get a good sense of that.  At any rate, the discussion after
that outlined why checking for ECHILD is probably a mistake in most
contexts, even if it is not in this specific instance.


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

On Wed, Nov 11, 2020 at 08:50:27PM -0600, Derek Martin wrote:
> On Wed, Nov 11, 2020 at 02:34:57PM -0600, Derek Martin wrote:
>   Fixing this almost certainly won't fix Matthias' problem, but may at
>   least help make it clear why it's happening.  There's also a bit of
>   trickiness in what to do about some of the cases...  But in this
>   instance, basically Mutt should report to the user that the process
>   was killed by a signal, what the signal was, and indicate that it
>   can't know if sending the message was actually successful.

Sorry, I confused myself here. :)  It's true that the current
implementation can't detect children killed by signals, but that's not
what's going on in this case.  AFAICT, NO process is actually killed
by a signal.  The issue was that Mutt got tired of waiting for
sendmail and exited.  But it has not provided itself a way to know
or report that, either.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

On Thu, Nov 12, 2020 at 01:20:06AM +0100, Oswald Buddenhagen wrote:
> On Wed, Nov 11, 2020 at 02:34:57PM -0600, Derek Martin wrote:
> > On Sat, Nov 07, 2020 at 01:06:00PM +0100, Matthias Apitz wrote:
> > > Does whatever the server is doing affect the mutt process too,
> > > despite mutt setting its own handler and blocking SIGCHLD during the
> > > fork/exec?
> > 
> > No, it does not.  Mutt will start with the default signal
> > dispositions.  Signal handler changes are copied across fork(), but
> > NOT across exec().
> > 
> not _entirely_:
> 
>   During an execve(2), the dispositions of handled signals are reset to
> the default; the dispositions of ignored signals are left unchanged.
> 
> (from the notes in linux' sigaction(2) man page.)
> 
> i don't think that changes the conclusion in this case, though.

Good catch!

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

On Wed, Nov 11, 2020 at 02:34:57PM -0600, Derek Martin wrote:
> [...] I would also agree that Mutt's exit status handling is
> incomplete.  That probably needs to be fixed.  I'd need to review
> the code in more detail to be confident about whether
> blocking/ignoring SIGCHLD is appropriate or not.

So I've just had a look at this, and I have some comments.  A proper
fix would be...complicated.  Bottom line, Mutt's code IS definitely
causing part of the headache, and I'd hoped to provide a proposal to
fix it with this message, but it would take more of a rework than I
have time for.

THAT SAID, my read of the code strongly suggests my initial guess
about what's wrong with Matthias' case was correct.  That means fixing
this bug won't actually fix that problem--it can only make it clearer
to the user what happened--and it also means that Matthias should be
able to fix the problem by fixing the broken sendmail script Mutt is
calling.

Specifically, Mutt uses alarm() to stop waiting too long for a rogue
sendmail process, and the mutt-users post indicated that SIGALRM was
what stopped waitpid() from continuing to wait.  So, the script that
Mutt launched never completed within SendmailWait seconds, whatever
that is, and Mutt decided to stop waiting for it.  SIGALRM doesn't
just happen at random--that's really the only plausible explanation
and it matches the code.  The rest of what I said then would
necessarily have happened:  The intermediate fork exited (see more
below about sendlib.c), which closed the script's STDIN, which caused
sendmail to get EOF reading the message and finally send it.  This
also explains why waitpid() was getting ECHILD: The child was in fact
not dead (see more below under background.c).

So to be clear, (I'm as certain as I can be without access to the code
that) Matthias' problem is in Mattias' code, not Mutt.  Mutt just
makes it worse/confusing by doing poor error handling.


Now, turning attention back to Mutt's code, a few notes about the bad,
the worse, and the ugly... :)

system.c:
  Mostly looks fine, BUT resets the disposition of SIGCHLD to SIG_DFL,
  which seems right, but differs from signal.c where it sets it to
  chld_handler (which just sets a flag).  See below.

signal.c:
  Sets the handler for SIGCHLD to chld_handler.  This is largely
  pointless.  It just sets SigChld = 1.  That's generally the correct
  way to handle a signal, but in this case it does nothing useful
  since Mutt calls waitpid all over the code.
  
  The only effect of setting SigChld is it triggers a redraw in
  update_bg_menu, but that can easily be eliminated by making the
  decision to do a redraw depend entirely on unconditionally calling
  mutt_background_process_waitpid, eliminating the SigChld check. In
  one place, mutt_background_compose_menu arguably abuses SigChld (the
  global flag, not the signal) to force a redraw.  Not exactly
  incorrect...  just a little gross, and completely unnecessary.

  Plus, unless I missed something, it appears that if _mutt_system()
  is ever called, this stops happening anyway, since it resets
  SIGCHLD's disposition to SIG_DFL (the signal handler will no longer
  ever execute), which likely proves that the above suggestion is
  already working.   And with SIGCHLD set that way, there's no cause
  to block it.  This is The Right Thing™.  

background.c:
  background_edit_landing_page() has this code:

while (!done)
{
  wait_rc = waitpid (bg_pid, NULL, WNOHANG);
  if ((wait_rc > 0) ||
  ((wait_rc < 0) && (errno == ECHILD)))
  {
rc = 0;
break;
  }
...

  Checking for ECHILD is unnecessary if children are handled properly.
  Or, said another way, if you're not ignoring SIGCHLD (i.e. SIG_IGN)
  then the only way you can get ECHILD is if the child IS NOT DEAD.
  In this context (background editing), that scenario makes no sense.  
  Since you're not ignoring your children, they will zombify until you
  ask for their status, so their PID can not be reused, and they will
  still be in the process table until you call waitpid().  The only
  other potential way you could have this happen is if you called
  waitpid() twice on the same pid.  Don't do that. :)

  This also eats the exit status of the editor, so mutt can't do
  anything intelligent with it later (like report that the editor
  failed/crashed).

  Worse, mutt_background_compose_menu() calls waitpid and doesn't even
  check the return value.  Not sure what the point of this is, but I'd
  assume the user would want to know if that process failed or
  crashed, and that's not happening.  

sendlib.c:
  Properly handling the return from waitpid() is a little tricky, and
  like most code I've seen, this code doesn't.  This looks like the
  cause of the behavior Matthias saw, though it probably won't happen
  if there isn't also something wrong with his send script.  The
  NATURE of the bug is that it causes Mutt to misreport t

Re: Mutt && handling of SIGCHLD while starting $sendmail

[Derek Martin]

Hello Matthias,

On Sat, Nov 07, 2020 at 01:06:00PM +0100, Matthias Apitz wrote:
> For more information, and before replying, please read the thread 
> http://lists.mutt.org/pipermail/mutt-users/Week-of-Mon-20201102/002057.html
> (even if the Subject: is wrong, because I started with this Subject:
> before looking deeper into the process chain and sources).

For what it's worth, by linking to an entire thread, much of which is
not really relevant to a potential Mutt bug, you're making the people
trying to help you do all the work to wade through all the irrelevant
bits and reassemble all the relevant context when we reply.  I think
that's bad form.  It would be much preferable if you instead posted a
new message that succinctly described what you are doing and how it is
breaking, so we can reply directly to that and quote the relevant
context.  That makes more work for you, but you're the one who needs
help, and we are not paid for our time...

That out of the way, a few points:

In the thread, you state:

> El día jueves, noviembre 05, 2020 a las 10:17:25a. m. -0800, Kevin J. 
> McCarthy escribió:
> > As far as I know, that mode of operation isn't supported by Mutt.  I'm 
> > confused how it is working, to be honest.  Mutt expects either $sendmail 
> > or $smtp_url to be set, and it controls the sending itself.
> 
> It *is* supported by mutt using the following trick: one of the arguments
> to mutt is:
> 
>  | mutt -d4 -e "set sendmail=\"cat --\"" ... | sendmail -t

Having to use a hack rather means that it is NOT supported--only that
doing arbitrary other unsupported things with hacks is supported. =8^)
You should not think of Mutt as supporting this--it does not.

> *** 2310,2315 
> --- 2310,2317 
> 
> if (pid != -1 && waitpid (pid, , 0) > 0)
>   st = WIFEXITED (st) ? WEXITSTATUS (st) : S_ERR; /* return child status 
> */
> +   else if (errno == ECHILD)
> + st = S_BKG;
> else
>   st = S_ERR;   /* error */

I agree with Oswald that this is wwrong.  I haven't looked at the Mutt
code here yet (well, I have, but not in a very long while) but I hope
to do that tonight.  I'm pretty sure I've commented before about
Mutt's child handling code... but it would have been years ago and I
need to refresh the context.  From the above snippet, I would also
agree that Mutt's exit status handling is incomplete.  That probably
needs to be fixed.  I'd need to review the code in more detail to be
confident about whether blocking/ignoring SIGCHLD is appropriate or
not.

Also there's a difference between blocking and ignoring, and that
likely matters.  The default disposition of SIGCHLD is to do nothing,
and here, that's not the same thing as SIG_IGN.  The latter tells the
OS that the parent doesn't care about its children's exit status (and
won't try to fetch them), so don't save them, which is one way your
program can get ECHILD.  Mutt wants the SIG_DFL behavior, and TBH
probably should lose the signal handler entirely.  Well-behaved
programs should not use signals whenever there is another way to do
it...

And to address Kevin's comment in that thread:

> Does whatever the server is doing affect the mutt process too,
> despite mutt setting its own handler and blocking SIGCHLD during the
> fork/exec?

No, it does not.  Mutt will start with the default signal
dispositions.  Signal handler changes are copied across fork(), but
NOT across exec().

Then finally, from this:

> 10502 alarm(0)  = 0
> 10502 rt_sigaction(SIGALRM, {sa_handler=0xd76dd3, sa_mask=[], 
> sa_flags=SA_RESTORER|0x200, sa_restorer=0x7f9762c035a0}, NULL, 8) = 0
> 10502 kill(10499, SIG_0)= 0
> 10502 exit_group(127)   = ?

I conclude that your process chain is exiting only due to a signal
(SIGALRM), which most likely indicates a bug on your end.  That
seems to be triggering the bug with Mutt's incomplete child management
code.  We have no insight into what your code is doing.

If I had to guess, my guess would be that whatever in your script is
dumping the message to stdout never actually completes; then it's
killed by SIGALRM due to timeout, causing file descriptors to be
closed, which causes sendmail to get EOF, and finally complete sending
the message.  That would, in all likelihood, also mean that Mutt is
technically correct--at the time its child dies, the message has in
fact not been sent.  It is only sent after the fact, due to the OS
being pretty good at cleaning up messes on behalf of user programs.
But that time delta is so small that it's hard for the humans to
notice the difference.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Tue, Aug 18, 2020 at 06:34:27AM -0400, Remco Rijnders wrote:
> On Mon, Aug 17, 2020 at 10:37:57PM -0500, Derek wrote in
> <20200818033757.gd28...@bladeshadow.org>:
> > It's worth pointing out one additional point that I haven't:  The
> > data in an attachment might be considered sensitive by the person
> > sending you the attachment, whereas to YOU it doesn't matter in the
> > slightest.  By relaxing your umask, you're reducing the security of
> > all of your senders' data, BY DEFAULT, without their knowledge or
> > consent.
> 
> That seems a bit far fetched to me. If you don't want your precious
> attachment to make it out to the public, don't send it. Once you hit
> send, it is out of your hands, a umask in mutt is not going to make
> a difference there.

Here, you are displaying your ignorance of threat models.  The user
may, in practical terms, have no choice, because sending you their
(or your) sensitive data may be a requirement for some formal or
informal transaction that they (or you) are about to perform.  It
could be something that they are doing on your behalf, or vice versa.
Or it could be info that one of you is providing the other on behalf
of a third party, on whose behalf you are acting as agent, perhaps in
the legal sense of the word, or perhaps informally. E-mail is a bad
choice for such a transfer and much better options do exist, but it is
still somewhat commonly used because it is ubiquitous and convenient.

There's also a huge difference between intentionally, maliciously
sharing someone's data, and doing so inadvertently, unintentionally,
simply because you've made a poor choice about how to configure your
tools.  You probably trust your mom not to give out your banking info
to random strangers.  Unless your mom happens to specialize in secure
software development or application security administration, I imagine
you DON'T trust her to understand how to configure her mail client
securely.

Although, it's worth pointing out that many victims (of all sorts) are
attacked by their own family members, whom they trusted.  Even in
those cases, it should require action on your part--not default action
on Mutt's part--that gets you owned.  It should never be possible to
BLAME MUTT for this.

> > > The difference between us is basically that I prefer that the user
> > > have the *possibility* to do it in the "wrong" way after being
> > > warned about consequences. Of course sane/secure defaults are a
> > > must.
> > 
> > Have you not been paying attention to the heat that Facebook and other
> > sites have been taking for their lack of care in protecting users'
> > sensitive data?
> 
> That is not a good analogy.

Of course it is.  In both cases the user is entrusting some level of
their data security to the policies and practices of a third party
(either Facebook, or mutt-dev), largely because they are unqualified
and incapable of doing so for themselves.  Often the user realizes
this... in many cases the user does not realize this.  It is the
responsibility of those trained and experienced in security to make
sure that those who aren't don't have to, because history shows they
generally will not take the time to try to learn, and even when they
do, they will most often fail to understand what they have learned, or
the gravity of it in relation to their specific circumstances.

It's exactly the same for Facebook as it is for mutt-dev.

> Sacha was not proposing changing anything like that but just
> proposing a patch that would help them and perhaps make others life
> easier too; No defaults would be changed

Again that's just wrong.  A configured default is still a default.  If
you aren't given the opportunity to decide how securely to save the
file, then the exhibited behavior is the default.  Anyone who sets a
relaxed umask has created for themselves a default of insecure saved
attachments.  You can't know that there's sensitive data in it until
you see it, which requires saving it to disk.  The instant it is on
disk and readable by anyone but you, if there is a knowledgeable
attacker on your system who is targeting you it is already too late
for you to do anything about it.

> and I can't see how this would catch anyone unaware.

Again, that's the problem, as it so often is with security:  YOU can't
see it.  But if an attacker is aware of it, they sure as hell will
exploit it.  You can be unaware because you don't have the facts.  You
can also be unaware because you don't understand the facts that you
have, or lack adequate imagination as to how those facts can go wrong
for you.

Anyone who uses a relaxed umask has made a bad decision, period.  The
feature should therefore not exist, so that users do not have the
opportunity to make what can ONLY be a bad decision.  The fact that
YOU, or any particular developer or mutt user can't see that is not a
good enough reason to compromise Mutt's security, most especially when
the benefit in convenience of doing so is actually so small, as in

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Fri, Aug 14, 2020 at 11:14:27AM +0200, sacham...@s0c4.net wrote:
> Hi all,
> 
> thank you all for this long discussion. Your security concerns are
> clear, as is clearer the intended system-usage scenario (most of you
> have) and Mutt's role in there.

It's worth pointing out one additional point that I haven't:  The
data in an attachment might be considered sensitive by the person
sending you the attachment, whereas to YOU it doesn't matter in the
slightest.  By relaxing your umask, you're reducing the security of
all of your senders' data, BY DEFAULT, without their knowledge or
consent.

> The difference between us is basically that I prefer that the user
> have the *possibility* to do it in the "wrong" way after being
> warned about consequences. Of course sane/secure defaults are a
> must.

Have you not been paying attention to the heat that Facebook and other
sites have been taking for their lack of care in protecting users'
sensitive data?  In many cases they're blamed even when the user has set
their privacy policy to more permissive settings (they should have
made it clearer!), and the breach is their own fault.  And of course
it's great when your fellow users (e.g. facebook friends) victimize
you because you made it easy for them, by compromising your own
sensitive data, because you didn't understand how sensitive it
actually was:

  https://www.huffpost.com/entry/burglary-ring-targets-fac_n_712629
  http://www.cbsnews.com/news/facebook-friend-suspected-in-burglary/
  https://sileo.com/facebook-status-update-leads-to-robbery/
  https://abc7.com/archive/9482852/
  
https://bits.blogs.nytimes.com/2010/09/12/burglars-picked-houses-based-on-facebook-updates/
  
https://www.nydailynews.com/news/crime/california-woman-home-burgled-facebook-friend-cops-article-1.1737842
  https://www.getsafe.com/how-burglars-use-social-media/
  https://www.thesun.co.uk/living/4119821/10-share-facebook-crime-target/

etc. etc  The odds that YOU become a target might be low, but it
happens with increasing frequency, and it could be catastrophic.  Your
odds go up significantly if someone shady in your circle learns that
you're an easy target due to careless security practices (which tend
to be habitual, and attackers are watching for signs).  Now imagine
instead of your vacation plans, it was your 401k account info that
some well-meaning idiot at your financial institution included in an
attachment in an e-mail you weren't even expecting them to send you...
Life savings gone in an instant.

Do you really think, even if you explain all of this in as much detail
as I have to your users, that they'll even understand it all?
Probably not.  And that's if their eyes don't glaze over after your
first sentence or two as they stop listening. 

That, in large part, is why your bank and the company you work for
(etc.) impose a wide variety of security measures that annoy you and
don't give you a choice about it, and it is why strict attachment
permissions in your mailer should not be optional:  The vast majority
of users will happily disable them if they can, but not grasp the full
ramifications of doing so until and unless they are actually bitten by
it (and most users probably would never know how it happened, still
not understanding, were they to be bitten by it).  Security is hard,
and most people--even many otherwise well-informed technical
users--really don't get it.  Sometimes, as in this case, when you do
know better, and especially when the price of better security is so
small (an occasional chmod command), it's your responsibility to
decide for those who don't know better. 

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Thu, Aug 13, 2020 at 10:03:43PM +0200, Vincent Lefevre wrote:
> On 2020-08-11 22:29:41 -0500, Derek Martin wrote:
> > The subversion example is a special case of an application that you
> > use through a web server, that has its own security implications.
> 
> Wrong! Subversion does not need a web server.

I didn't say it was required, and it makes absolutely zero difference:
in your use case you're using your second user to run svnserve--a
network service--which is functionally equivalent to the webserver for
this use case.  This is standard security practice for a running
service.  You're not supposed to USE that user, and that's actually
the point of having the separate user...  If you are you may as well
have just used your normal user, because you're missing the point. 

Then, when you do your checkout, the files will be owned by and
accessible to your main user, not the second user that svnserve is
running as.  There is absolutely no cause for the files to be
group-accessible.  It's absolutely nothing like your silly contrived
attachment-sharing use case.

In fact the second user is not yours at all...  it belongs to the
subversion repository/service, much as the www-data user belongs to
the web server... not YOU.  You can see this more clearly if you
imagine a second human added to your system who will also access the
same repository using the same mechanism and user, and a separate
sysadmin who set this all up for you.  And I don't care if that's not
how you'd do it--you COULD, and it illustrates the point.

And TBH even the svnserve case is a bit silly (over-paranoid), it
provides very little (but yes, some) extra protection.  You'd do just
about as well if you never run unstable SVN code and never interact
with your repository any way other than by using svn commands. A
fairly easy way to avoid trashing it accidentally with something like
rm -rf is to put the repository in a very different path from your
home directory or other file store you commonly use, where you have no
reason to be running shell commands.

> > It's nothing like using multiple users on your system to do different
> > tasks, like reading your e-mail with one user, and then handling
> > attachments with a different one.
> 
> Correction: reading e-mail and handling attachments with one uid,
> but also being able to access saved attachments with another uid.

It's not a correction, those things are equivalent. It's still
superfluous because having multiple users that you're USING is just
idiotic.  Switching between users to get things done is going to get in
your way a lot more often than just group perms on e-mail attachments,
so if you want a configurable umask in Mutt because you're running
with this insane use case, for the diminutive boost of efficiency
gained by not having to run chmod on the attchments you actually want
to share, then you've missed the forest for the trees.

That's why absolutely no one actually does this, without also being
completely insane... or at least completely misunderstanding what
they're doing and why.

> Note: Even without a sandbox, one does not absolutely need to make
> attachments group accessible. The main account could do a SSHFS mount
> of the directory that contains the saved attachments. But allowing
> to control the permissions of the attachments inside Mutt would avoid
> the need for SSHFS.

Or just use chmod.  Or write a script with a one-character name that
you can run any time you want, to chmod all the files in your
attachments directory, *once you've determined the attachments in a
hard-coded directory have nothing sensitive in them*, if chmod is so
troublesome for you.  Or...stop being insane enough to have a use case
that's so contrived and completely ridiculous.

[I also really want to know who is sending/receiving all these copious
amounts of file attachments that occasionally running chmod on the
ones you actually want to share with someone on the same system is
such a bother...]

Bottom line:  Sharing potentially sensitive data that came to your
inbox should ALWAYS be a forced choice, so that you don't accidentally
make your banking info or medical history etc. available to everyone
on your system, or your raise and bonus info that your boss just
e-mailed you gets shared with your coworkers (or whatever).  It should
NEVER be the default, and you're just plain wrong if you think it
should be, even by your own choice.  If there's no one else on your
system, and you're not doing something completely idiotic, then it's
entirely a non-issue; and if there are other users on your system, you
DO NOT WANT THIS, even if you think you do.  You can do it and it will
be totally fine... right up until that one time it isn't, at which
point you'll finally understand this.  Michael did, which is why he
wrote it that way.  Thomas did, which is why he adamantly and
explicitly protected Michael's decision (when at the time, I

Re: Taking a break for a bit

[Derek Martin]

On Wed, Aug 12, 2020 at 11:29:06PM +0200, Oswald Buddenhagen wrote:
> On Wed, Aug 12, 2020 at 03:41:50PM -0500, Derek Martin wrote:
> > On Wed, Aug 12, 2020 at 03:16:46PM +0200, Oswald Buddenhagen wrote:
> > > meanwhile, kevin has been actively hostile towards changes he didn't
> > > deem "minimal enough".  that's a reliable way to make sure the code
> > > base is stale, rather than stable.
> > 
> > I don't think that's quite a fair characterization either,
> > 
> i just went through about three years of list activity (mid 2016 to mid
> 2019), and have a somewhat solid recollection of recent activity, so i'm
> *quite* confident about that characterization.

I can't speak for what's in Kevin's head, but I'm just suggesting that
you may be mistaking the symptom for the disease... In other words,
perhaps its not that the change isn't minimal that he's being
"hostile" about (and TBH I can only really remember one time I thought
Kevin was actually being hostile _at all_, which he and I actually
discussed off list), but rather some unspoken other thing that just
manifests that way.

> mutt's design philosophy more or less boils down to "minimal features,
> maximal configurability+extensibility", and it makes sense to be
> conservative about it (within reason), especially when redundant features or
> outright misfeatures are being proposed.

I would phrase that differently, but I think it's fair.

> but within that framework there is still a lot of room for different
> *development* philosophies, and that's where mutt needs improvement.

Maybe, but philosophy and practice often have big gaps.  I wanted to
call out in my previous message that in the past, it was often the
case that patches were rejected NOT ONLY because they were considered
inadequate quality, but because in addition, neither the author, nor
anyone else, wanted to work on them further to improve the quality to
the point where they would be considered acceptable. I believe the
sidebar patch and the NNTP patch were both in that category--it was
Kevin who finally reworked the former, along with some requisite
underpinnings in Mutt, and finally included it.   Some patches that
provided useful functionality would have been good candidates for
inclusion, but they weren't up to snuff, and no one was willing to put
in the extra effort to get them there.  It wasn't a matter of
philiosophy, but of elbow grease.

> > >   also, don't let derek have the last word too often, because,
> > >   duh.
> > > :-D
> > 
> > I'll point out that [...]
> > 
> dude, chill - if i was really serious about that, you'd be hearing from me
> much more often. also, there would be no huge smiley above. ;-)

Sure, I got that, but just because you're joking, doesn't mean you're
not also serious. =8^)

But FWIW I'm just saying I'm well aware I'm pretty conservative about
accepting patches generally, but that that's in keeping with the
original philosophy that drove Mutt's creation, both in general, and
in the specific. I think that perspective is very valuable, since it
led to us all having the generally excellent mail client we're
discussing here, so I will share it.  Passionately. :)

Over the years I've watched all manner of software projects change
things that were good, to bad, on a small scale or large, mostly
because no one involved at the time remembered/understood why it was
that way to begin with, and didn't realize what they were breaking.
Well, I do. :)  [Sometimes they just don't care, and there's nothing
for that.]

> you do however have a tendency to make an elephant out of a mole
> hill when it comes to the investment cost of changes, completely
> ignoring the opportunity cost of *not* doing them - and that is
> symptomatic of the culture here.

Obviously I disagree.  I always consider what I think the benefit
would be, as well as whether or not there is another way to achieve
the same effect already (either within or without Mutt) that makes the
feature superfluous.  I just usually estimate the value is low.  :)  I
may undervalue the benefit of a given feature--but that is highly
subjective.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Taking a break for a bit

[Derek Martin]

On Wed, Aug 12, 2020 at 03:16:46PM +0200, Oswald Buddenhagen wrote:
> On Wed, Jul 29, 2020 Derek wrote:
> > I'll be honest--what I'd really prefer to see is the two projects merge,
> > where "NeoMutt" would be the place were things were tried, fleshed out,
> > and refined so that they could be included in Mutt.
> > 
> let me remind everyone that this is pretty much exactly what neomutt started
> out as.

That may have been Rich's intention, but it was clear from the outset
that it was never going to be that, because from the start he wanted
to apply a bunch of patches that had been rejected by every Mutt
maintainer to that point, mostly for good reason.  And while it's not
an insurmountable technical obstacle, it probably didn't help that he
wanted to use Git for SC instead of Mercurial, which Mutt used at the
time... [I've never had a horse in that race, and Git has obviously
won since then.]

> meanwhile, kevin has been actively hostile towards changes he didn't
> deem "minimal enough".  that's a reliable way to make sure the code
> base is stale, rather than stable.

I don't think that's quite a fair characterization either, but every
change needs to be reviewed, and larger changes are a much bigger
burden in that regard.

> in particular:
> - give timely and detailed feedback. you're actually good at this.

Much better than Michael or Thomas were, to be sure.  And a bit better
than Brendan.

> - don't be so conservative about the types of changes you accept, in
> particular when it comes to internals.

You have to remember where Mutt came from:

  “All mail clients suck. This one just sucks less.”— me, circa 1995

[For those for whom this is not clear, the attribution "me" is not the
personal pronoun "me" but rather the initials of Mutt's author, Michael
Elkins.]

In large part its conservatism is WHY it sucked less (and mostly still
does) than most other clients, as that was the main driving motivator
for what went into Mutt (and still is to large extent).  Adding
features willy-nilly or accepting patches whose quality is dubious
won't allow you to keep a reputation of sucking less for very long.  
It was quite trying to get Michael or Thomas to accept patches that
were genuinely deserving--the same is not true of Kevin... so long as
he, or someone he trusts, actually has time to commit to reviewing the
change.  And that does matter.

>   also, don't let derek have the last word too often, because, duh. :-D

I'll point out that nothing I've shot down on this list in even
remotely recent memory has not already been rejected for inclusion by
all of the previous maintainers who'd seen it (including Kevin, in at
least some cases).  Whether or not you agree with me, THEY all did (or
else we'd have had a configurable umask decades ago, for example), and
our collective wisdom has to count for something.  Our adamant
opposition to such features always has had as its goal the maintenance
of Mutt's reputation to suck less, and not just for its reputation's
sake...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Wed, Aug 12, 2020 at 02:40:16AM +0200, Vincent Lefevre wrote:
> On 2020-08-06 18:40:50 -0500, Derek Martin wrote:
> > Are you serious, Vincent?  I'm pretty sure you well know that this is
> > a horrible idea, clearly contrary to best security practices, that no
> > competent sysadmin managing servers holding anything vaguely sensitive
> > would ever allow on a multi-user system (and we've already established
> > that systems only ever used by one human render the configurable umask
> > moot). This is system security 101 (e.g. SANS GSEC). Users to
> > usernames are 1:1.
> 
> This is complete nonsense. 

I agree; everything you said that followed IS complete nonsense.
The subversion example is a special case of an application that you
use through a web server, that has its own security implications.
It's nothing like using multiple users on your system to do different
tasks, like reading your e-mail with one user, and then handling
attachments with a different one.  In 35+ years of computing, a
significant chunk of which I was a professional sysadmin/security
admin, I've seen exactly zero people do that.  That's not a very
believable scenario. There's no sane reason to do this, and Mutt
should not attempt to cater to insanity.  

> I am the sysadmin and the only user of my own machines.

I addressed this case in the post I linked.  If this is the case YOU
DON'T NEED UMASK, even with your completely silly, contrived example.
Because as you yourself said, you use the user that makes sense to use
for the given situation, which in this case would be the user which
already has access to the files.

THIS IS A BAD FEATURE.  It will either provide NO BENEFIT or ACTUAL
DETRIMENT to anyone who wants to use it.  Mutt should not implement
it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Fri, Aug 07, 2020 at 02:31:01PM +0200, Steffen Nurpmeso wrote:
> Derek Martin wrote in
>  <20200806234050.gb8...@bladeshadow.org>:
>  |On Fri, Aug 07, 2020 at 12:56:34AM +0200, Vincent Lefevre wrote:
>  |> On 2020-08-06 10:50:23 -0500, Derek Martin wrote:
>  |>> On Wed, Jul 29, 2020 at 12:55:07PM -0500, Derek Martin wrote:
>  |>>> On Tue, Jul 28, 2020 at 08:03:23PM +0200, sacham...@s0c4.net wrote:
>  |>>>> The thread, and even older threads referenced there, is from 2007.
>  |>>>> Since then, the security field have evolved - now we have SeLinux,
>  |>>>> Apparmor and other techniques which are capable to provide even
>  |>>>> better security than umask(077)
>  |>>> 
>  |>>> None of those changes affect this issue in any meaningful way.
>  |>>> SELinux predates that thread by at least two years (longer, though it
>  |>>> was not generally available to the public until ~2005).  The arguments
>  |>>> made in those threads still stand, and I will not repeat them here.
>  |>> 
>  |>> And FWIW, here's a more precise and detailed description I posted MUCH
>  |>> more recently than 2007, which explains why this is a bad idea.
>  |>> Everything here remains true, regardless of any evolution you think
>  |>> has happened in the security world.
>  |>> 
>  |>> https://www.mail-archive.com/mutt-users@mutt.org/msg49810.html
>  ...
>  |Are you serious, Vincent?  I'm pretty sure you well know that this is
>  |a horrible idea, clearly contrary to best security practices, that no
>  ...
>  |> On such a system using umask (007) for secondary ids seems logical
>  |> and safe.
>  |
>  |No, it doesn't. Even if someone were to run Mutt on such a horribly
>  |mismanaged system, its system security is generally suspect, so it is
>  |even more important for Mutt to make sure the files are never saved
>  |readable by anyone other than the user who created them.  Regardless,
>  |Mutt should stick to its guns concerning maintaining the security of
>  |its users' files.
>  |
>  |And remember, what we're trading here is the, what, 3 seconds it takes
>  |for the user to type "chmod 644 *" (or similar) if they really want to
>  |do this.  It's a small price to pay for the best insurance available
>  |that no one is ever able to read your sensitive mail attachments
>  |without you explicitly taking action to allow them to.  You'll never
>  |be able to blame Mutt for this.
> 
> I take that bait. As an outsider i nonetheless think this is very
> much of an over- reaction.  Users have an umask, and they usually
> have it for a reason.

You obviously didn't read the post.  It explains why the umask for
mail clients should not be treated the same as for everything else the
user does.

> I do not really see a security threat,

And that is the problem.  But this also is explained in the post.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Fri, Aug 07, 2020 at 12:56:34AM +0200, Vincent Lefevre wrote:
> On 2020-08-06 10:50:23 -0500, Derek Martin wrote:
> > On Wed, Jul 29, 2020 at 12:55:07PM -0500, Derek Martin wrote:
> > > On Tue, Jul 28, 2020 at 08:03:23PM +0200, sacham...@s0c4.net wrote:
> > > > The thread, and even older threads referenced there, is from 2007.
> > > > Since then, the security field have evolved - now we have SeLinux,
> > > > Apparmor and other techniques which are capable to provide even
> > > > better security than umask(077)
> > > 
> > > None of those changes affect this issue in any meaningful way.
> > > SELinux predates that thread by at least two years (longer, though it
> > > was not generally available to the public until ~2005).  The arguments
> > > made in those threads still stand, and I will not repeat them here.
> > 
> > And FWIW, here's a more precise and detailed description I posted MUCH
> > more recently than 2007, which explains why this is a bad idea.
> > Everything here remains true, regardless of any evolution you think
> > has happened in the security world.
> > 
> > https://www.mail-archive.com/mutt-users@mutt.org/msg49810.html
> 
> Perhaps the OP is on a system where each physical user is in his
> own group, but possibly with several user ids: the main id would
> typically use mode 0600 for all private files, but the secondary
> ids would use mode 0660, i.e. the main id can access the private
> files of all ids of the user, but the secondary ids would be able
> to access only their private files.

Are you serious, Vincent?  I'm pretty sure you well know that this is
a horrible idea, clearly contrary to best security practices, that no
competent sysadmin managing servers holding anything vaguely sensitive
would ever allow on a multi-user system (and we've already established
that systems only ever used by one human render the configurable umask
moot). This is system security 101 (e.g. SANS GSEC). Users to
usernames are 1:1.

Just the complexity of managing all those extra usernames increases the
likelihood that one or more of them is misconfigured, or that the
files they own are improperly protected, allowing inadvertent access
to someone's sensitive files. It also greatly increases the likelihood
that the user will "lend" one of their "spare" usernames to someone
who shouldn't have it, not realizing what it allows them to do (or
see).  And for all those reasons and others it increases the
complexity of auditing, with certainty, who is doing what on the
system that might be compromising its security.  

> On such a system using umask (007) for secondary ids seems logical
> and safe.

No, it doesn't. Even if someone were to run Mutt on such a horribly
mismanaged system, its system security is generally suspect, so it is
even more important for Mutt to make sure the files are never saved
readable by anyone other than the user who created them.  Regardless,
Mutt should stick to its guns concerning maintaining the security of
its users' files.

And remember, what we're trading here is the, what, 3 seconds it takes
for the user to type "chmod 644 *" (or similar) if they really want to
do this.  It's a small price to pay for the best insurance available
that no one is ever able to read your sensitive mail attachments
without you explicitly taking action to allow them to.  You'll never
be able to blame Mutt for this.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Wed, Jul 29, 2020 at 12:55:07PM -0500, Derek Martin wrote:
> On Tue, Jul 28, 2020 at 08:03:23PM +0200, sacham...@s0c4.net wrote:
> > The thread, and even older threads referenced there, is from 2007.
> > Since then, the security field have evolved - now we have SeLinux,
> > Apparmor and other techniques which are capable to provide even
> > better security than umask(077)
> 
> None of those changes affect this issue in any meaningful way.
> SELinux predates that thread by at least two years (longer, though it
> was not generally available to the public until ~2005).  The arguments
> made in those threads still stand, and I will not repeat them here.

And FWIW, here's a more precise and detailed description I posted MUCH
more recently than 2007, which explains why this is a bad idea.
Everything here remains true, regardless of any evolution you think
has happened in the security world.

https://www.mail-archive.com/mutt-users@mutt.org/msg49810.html

I think we (the Mutt developer community) should consider this the
canonical argument for why umask should NEVER be configrable.  This
does come up often, so I wonder if it's not worth making this a FAQ
entry on the website...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Add support for DT_R(UN)PATH in ELF executables.

[Derek Martin]

On Tue, Jul 28, 2020 at 12:59:55AM +0200, Mono DHS wrote:
> Subject says it all 

What is the motivation for this patch?

My initial reaction is this patch adds a bunch of configure code that
will be used by pretty close to no one, so I don't see the value.  It
will surprise no one who's been around here for any length of time
that I would not support such patches...

I pretty much feel the same way about all the other patches you
submitted.  At the very least I think some discussion of why they are
useful is warranted...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.


> From 5d3623986a1dac95ce517cee0048520eec10800f Mon Sep 17 00:00:00 2001
> From: Mono DHS 
> Date: Sun, 19 Jul 2020 16:57:38 +0200
> Subject: [PATCH] Add support for DT_R(UN)PATH in ELF executables.
> 
> ---
>  configure.ac | 73 
> 
>  1 file changed, 73 insertions(+)
> 
> diff --git a/configure.ac b/configure.ac
> index 
> 7906ce35b66d07845495377fd6bb767375a9bd88..a0554460f77938c26ba99e63078b4b5d72e0e5d2
>  100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -119,6 +119,11 @@ AH_BOTTOM([/* fseeko portability defines */
>  MUTT_C99_INTTYPES
>  AC_TYPE_LONG_LONG_INT
>  
> +AC_ARG_ENABLE(rpath,
> +AS_HELP_STRING([--enable-rpath],[Set DT_R(UN)PATH in the ELF 
> executable]),
> +[],
> +[enable_rpath=no])
> +
>  ac_aux_path_sendmail=/usr/sbin:/usr/lib
>  AC_PATH_PROG(SENDMAIL, sendmail, /usr/sbin/sendmail, 
> $PATH:$ac_aux_path_sendmail)
>  AC_DEFINE_UNQUOTED(SENDMAIL,"$ac_cv_path_SENDMAIL", [Where to find sendmail 
> on your system.])
> @@ -133,6 +138,11 @@ AC_ARG_WITH(sqlite3,
>  if test x$with_sqlite3 != xno; then
>if test x$with_sqlite3 != xyes; then
>  LDFLAGS="$LDFLAGS -L$with_sqlite3/lib"
> +if test x${enable_rpath} = xyes; then
> +  if test x${with_sqlite3} != x/usr; then
> +LDFLAGS="${LDFLAGS} -Wl,-rpath,${with_sqlite3}/lib"
> +  fi
> +fi
>  CPPFLAGS="$CPPFLAGS -I$with_sqlite3/include"
>fi
>saved_LIBS="$LIBS"
> @@ -299,6 +309,11 @@ main ()
>  CPPFLAGS="$CPPFLAGS -I${withval}/include"
>  fi
>  LDFLAGS="$LDFLAGS -L${withval}/lib"
> +if test x${enable_rpath} = xyes; then
> +  if test x${withval} != x/usr; then
> +LDFLAGS="${LDFLAGS} -Wl,-rpath,${withval}/lib"
> +  fi
> +fi
>  fi
>  fi
>  AC_MSG_RESULT($mutt_cv_slang)
> @@ -325,6 +340,9 @@ main ()
>  fi
>  if test x$mutt_cv_curses != x/usr; then
>  LDFLAGS="$LDFLAGS -L${mutt_cv_curses}/lib"
> +if test x${enable_rpath} = xyes; then
> +  LDFLAGS="${LDFLAGS} 
> -Wl,-rpath,${mutt_cv_curses}/lib"
> +fi
>  CPPFLAGS="$CPPFLAGS -I${mutt_cv_curses}/include"
>  fi])
>  
> @@ -767,6 +785,11 @@ then
>   if test "$zlib_prefix" != "yes" -a "$zlib_prefix" != "auto"
>   then
> LDFLAGS="$LDFLAGS -L$zlib_prefix/lib"
> +   if test x${enable_rpath} = xyes; then
> + if test x${zlib_prefix} != x/usr; then
> +   LDFLAGS="${LDFLAGS} -Wl,-rpath,${zlib_prefix}/lib"
> + fi
> +   fi
>CPPFLAGS="$CPPFLAGS -I$zlib_prefix/include"
>   fi
>  saved_LIBS="$LIBS"
> @@ -800,6 +823,11 @@ AC_ARG_WITH(ssl, 
> AS_HELP_STRING([--with-ssl@<:@=PFX@:>@],[Enable TLS support usi
>  if test "$with_ssl" != "yes"
>  then
>   LDFLAGS="$LDFLAGS -L$withval/lib"
> + if test x${enable_rpath} = xyes; then
> +  if test x${withval} != x/usr; then
> +   LDFLAGS="${LDFLAGS} -Wl,-rpath,${withval}/lib"
> +  fi
> + fi
>   CPPFLAGS="$CPPFLAGS -I$withval/include"
>  fi
>  saved_LIBS="$LIBS"
> @@ -843,6 +871,11 @@ then
>  if test "$gnutls_prefix" != "yes"
>  then
>LDFLAGS="$LDFLAGS -L$gnutls_prefix/lib"
> +  if test x${enable_rpath} = xyes; then
> +if test x${gnutls_prefix} != x/usr; then
> +  LDFLAGS="${LDFLAGS} -Wl,-rpath,${gnutls_prefix}/lib"
> +fi
> +  fi
>CPPFLAGS="$CPPFLAGS -I$gnutls_prefix/include"
>  fi
>  saved_LIBS="$LIBS"
> @@ -889,6 +922,11 @@ AC_ARG_WITH(sasl, 
> AS_HELP_STRING([--with-sasl@<:@=PFX@:>@],[Use SASL network sec
>then
>  CPPFLAGS="$CPPFLAGS -I$with_sasl/include"
>  LDFLAGS="$LDFLAGS -L$with_sasl/lib"
> +if test x${enable_rpath} = xyes; then
> +  if test x${with_sasl} != x/usr; then
> +

Re: [PATCH] Support for overriding permissions of saved files

[Derek Martin]

On Tue, Jul 28, 2020 at 08:03:23PM +0200, sacham...@s0c4.net wrote:
> The thread, and even older threads referenced there, is from 2007.
> Since then, the security field have evolved - now we have SeLinux,
> Apparmor and other techniques which are capable to provide even
> better security than umask(077)

None of those changes affect this issue in any meaningful way.
SELinux predates that thread by at least two years (longer, though it
was not generally available to the public until ~2005).  The arguments
made in those threads still stand, and I will not repeat them here.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Fix man section in reference to mutt_dotlock

[Derek Martin]

On Sun, Jul 26, 2020 at 03:04:33AM +0300, Maxim Tarasov wrote:
> ---
>  init.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/init.h b/init.h
> index 06d4cc9a..f0d2c697 100644
> --- a/init.h
> +++ b/init.h
> @@ -920,7 +920,7 @@ struct option_t MuttVars[] = {
>{ "dotlock_program",  DT_PATH, R_NONE, {.p=}, {.p=BINDIR 
> "/mutt_dotlock"} },
>/*
>** .pp
> -  ** Contains the path of the \fCmutt_dotlock(8)\fP binary to be used by
> +  ** Contains the path of the \fCmutt_dotlock(1)\fP binary to be used by
>** mutt.
>*/
>  #endif

I think this is not correct.  Section 1 of the man pages is meant for
user-oriented utilities and applications, and mutt_dotlock is not such
a thing--users generally have no call to run it directly.  It is more
a helper program for the administration of locks, and section 8 is for
admin programs.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Taking a break for a bit

[Derek Martin]

On Wed, Jul 29, 2020 at 01:06:00PM +0100, Richard Russon wrote:
> > I'm going to take a few weeks off from Mutt development.
> 
> Thanks for all your hard work on Mutt.

Agree!

> > Five years later
> 
> You've been *maintaining* Mutt, but we've been *developing* NeoMutt.

First off I think that's neither a fair nor accurate assesment; under
Kevin, Mutt has seen more active development than as under previous
maintaners.

Secondly I think the problem remains, as it ever has been, that the
NeoMutt project has a fundamentally different philosophy than Mutt.
Historically NeoMutt has been generally in favor of integrating
patches which, while they may have provided desired (by some)
functionality, have been rejected by Mutt, because in large part the
code was kind of... bad, and typically there wasn't much interest in
improving it--just requests to include it as-is. Mutt has been much
more conservative about what it will accept, for pretty good reason:
Mutt currently has 36 open issues--most of which are feature
requests--whereas NeoMutt has ~5x as many, and a number of them look
to be serious bugs, at a glance.  While conservatism relaxed
/somewhat/ under Kevin's leadership, with a mind toward making sure
Mutt doesn't fall behind the curve, it seems to me the difference in
philosophy is still fundamentally true.

That said, I'll admit that I have not tracked NeoMutt development
recently.  Perhaps it is no longer true. I'll be honest--what I'd
really prefer to see is the two projects merge, where "NeoMutt" would
be the place were things were tried, fleshed out, and refined so that
they could be included in Mutt.  Though I suspect by now development
has diverged sufficiently to make that impractical.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change hardcoded subject of replies

[Derek Martin]

On Fri, Jul 24, 2020 at 06:56:38AM +0200, Claus Assmann wrote:
> On Fri, Jul 24, 2020, Tom Ryder wrote:
> 
> > > > +env->subject = safe_strdup ("Re:");
> 
> > Would you or Maxim consider making it a _("translatable string")---or
> > perhaps even better, a configurable one---for those who speak other
> > languages?
> 
> Other than Latin?
> "Re:" is a standard that should not be translated.

Except some languages don't use it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change hardcoded subject of replies

[Derek Martin]

On Thu, Jul 23, 2020 at 07:15:11PM +0300, Maxim Tarasov wrote:
> First, "Re: your mail" sounds like a spam mail heading and doesn't
> convey any useful information. 

True enough, but, the fault is with the original sender for not
providing a meaningful subject, and if they don't want to receive
responses that look like spam they should...not do that.

> Second, setting the subject to just "Re:" will enable the user to
> immediately fill in the rest according to the contents of their email
> if they want to. Otherwise "Re:" should be enough to signify that this
> is a reply to an email with an empty subject line.

TBH I never liked "Re: your mail" but leaving it empty seems no better
to me.  Most people probably will be inclined to ignore it (if they
even notice). I think some other mailers default to something like
"Re: (no subject)" which seems more suitable to me.  It avoids
perpetuating the blank subject and indicates to the original sender
that they failed to provide a meaningful subject.  I think you could
argue it's no less spammy than the other two options, but let's be
honest: If the sender didn't provide a meaningful subject, you're not
going to fix that other than manually.

> It might be tempting to extract this part into a configuration
> variable, but I doubt anyone would really welcome the opportunity to
> invent some generic "Re: ..." subject.

Why?  This is Mutt.  =8^)

TBH I don't feel very strongly about this, but I don't think leaving
it blank is better, and I do think that if we're going to bother to
change it, we should change it to something that is better.  And my
guess is what people will find more acceptable is purely a matter of
taste and the rules of etiquette in their local culture.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: locking mechanism

[Derek Martin]

On Mon, May 11, 2020 at 08:38:19PM +0200, Steffen Nurpmeso wrote:
> Vincent Lefevre wrote in
> <20200510204809.ga71...@zira.vinc17.org>:
>  |Related to commit 7bd57bc3c24adf97f1f57bd6bb2fd18347f8cbbd, is
>  |dotlocking still used nowadays?
> 
> I find yes.  Or at least last i looked, some MTAs aka MDA or
> whatever the right name is (LDA?  postfix
> (configurable), i think OpenBSDs mail.local (which saw heavy
> modifications lately though)) create these files, and then i think
> it seems sensible to embed in this locking strategy.

IIRC most of the MDAs support multiple locking mechanisms, and at
least some of them use multiple mechanism simultaneously.  But dot
locking is slow, which probably doesn't matter on your home e-mail
server, but would matter on a mail server that handles millions of
messages a day...  And it may still be unreliable, depending on the
exact mix of things you have.  If you have one mechanism that's known
to work reliably across your whole mail system, you should use that,
and hope that it's not dot locking.

>  |Let's recall that dotlocking alone is not safe with some file systems,
>  |such as NFS, since even if the client has an exclusive access to the
>  |mailbox file, there is no guarantee that the synchronization will be
>  |done before another program accesses the mailbox. In short, a working
>  |fcntl locking is needed. But if it is available, then dotlocking is
>  |useless. So, IMHO, if still supported, dotlocking should just be seen
>  |as a fallback, and if mutt_dotlock cannot be installed with the right
>  |permissions, the installation of Mutt should not fail.
> 
> I do not think so.  fcntl is "the new way"

I don't think anything that's been available for > 20 years in the
world of computing can be considered "new" anymore... =8^)

> and used anyway (for my MUA), but embedding into the scheme used by
> others is crucial, and as long as they use dotlock files for
> synchronization i will use it.

All MDAs that are still supported use POSIX locking.  So dot locking
is superfluous and only slows you down.

But let's not also forget that NFS is not the only problem.  There are
a wide array of file systems, and you can't be sure they implement
your favorite locking mechanism.   But, t's more likely that if POSIX
locking doesn't work on those, since it's one call and its interface
expects that it might not be implemented everywhere, you'll likely
just get EINVAL back, whereas a dot lock might be more likely to fail
silently (as it did under Linux).

> NFS locking is "now" ok i have heard (here "now" is maybe almost two
> decades),

That depends on what you mean:

If you're on Linux, and:

  - If you mean POSIX locking, yes.  It's worked reliably since at
least as far back as 2.4, and I believe earlier. (There was a
really old bug in lockd that broke it, but that's been fixed for
about 20 years or something)...  AFAIK there have been no reported
bugs with it since.

  - If you mean dot locking...  In Linux it's been "OK" since 2.6.5,
if all of your clients and server are running Linux and NFSv3 or
greater.  Maybe. :)

If you mean NOT Linux, or you have a mix of things...  file locking
over NFS is a horror show, in general, and if your environment is not
100% homogenous all bets are off.  There have been various bugs in the
NFS and/or locking implementations of pretty much every platform ever,
and they don't generally interoperate well.

If you have NFS in the picture, and your site isn't 100% homogenous,
you'd better know exactly what's supported reliably by all of them,
and it's pretty much guaranteed to be POSIX locking, by now--it's kind
of the point of POSIX.  Otherwise, you may as well assume data
corruption is guaranteed.  Better to avoid accessing your mail spool
over NFS, or use Maildir.

I'm not sure that it makes sense to remove dot locking from Mutt
entirely, but it probably does make sense to turn it off unless
explicitly requested.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Clarify CH_WEED debug message

[Derek Martin]

On Fri, May 08, 2020 at 06:59:18AM +0200, Petr Pisar wrote:
> On Fri, May 08, 2020 at 06:42:57AM +0200, Claus Assmann wrote:
> > On Thu, May 07, 2020, Remco Rijnders wrote:
> > 
> > > -  dprint (1, (debugfile, "WEED is %s\n", (flags & CH_WEED) ? "Set" : 
> > > "Not"));
> > > +  dprint (1, (debugfile, "WEED is %s\n", (flags & CH_WEED) ? "Set" : 
> > > "Not Set"));
> > 
> > Save some bytes:-)
> > 
> > dprint (1, (debugfile, "WEED is %sSet\n", (flags & CH_WEED) ? "" : "Not "));
> 
> The message probably does not get translated, but from point of view of
> localization, please do not glue sentences from words.

Can you not just translate both parts?


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Use curses' raw instead of cbreak mode to capture \C[cyz\]

[Derek Martin]

On Wed, May 06, 2020 at 11:44:06AM +0200, Christopher Zimmermann wrote:
> Hello dear mutt developers,
> I prepared a merge request that needs some consideration.
[...]

Surprising absolutely no one I'm sure, I agree with Kevin.  You
already found the correct solution to your problem (use stty to unbind
or change the terminal's interpretation of ctrl-y), and generally
people who use console-based programs tend to know and expect those
behaviors.

If you'd just posted your problem on mutt-users the first place, you'd
likely have had an answer in minutes how to fix it.  :)  There are
probably dozens of people on mutt's mailing lists who understand the
issue and recognize it immediately.  Half way through reading your
post I was going to reply with something like "Just use stty to change
it..." until I saw you already figured it out.

Mutt's behavior here doesn't need to be changed, and doing so will
likely surprise people, causing the exact opposite confusion to yours,
but one that is unexpectedly mutt-specific.  Seems like a bad idea to
me.

> Users should not need to know such deep intrinsics of terminals
> these days.

First of all they're hardly deep... stty is terminals 101.  Pick up
any Linux/Unix primer, and look in the section on terminals or
terminal applications, and I can just about guarantee there will be a
discussion about stty and how certain terminal control behaviors are
bound to key sequences with it (as well as a number of other things it
can do).  If it doesn't, it's not a very good primer.  It's part of
basic terminal operation.  And if you've been using Unix for 20 years,
how is it possible you've never encountered the backspace problem?
That was a problem virtually everywhere for the better part of a
decade.  [In case it's not obvious, the solution to most of the easier
variants of that problem is stty erase--though it did get rather
annoyingly complicated for the not-so-simple cases.]

But even if such knowledge was deep...  Why?  If you're going to make
heavy use of something, how is it ever in your best intrest to
intentionally remain completely ignorant of how it works?  I think the
only type of user for whom that statement makes sense is one who will
never use a terminal application, of which there certainly are some,
even on Unix systems.  Then for sure, learning how terminals work is a
complete waste of that person's time, unless they happen to find it
interesting.

If it's never affected you before, you should consider yourself lucky.
If I cared to spend the time to dig them up I could point you at
probably a dozen or more bugs I've filed or commented on over the
years that were related to terminal behaviors, most of which were
caused by some well-intentioned but misguided change in configuration
or coding approach on the part of some package maintainer, who didn't
really understand why it was the way it was before.

Blech.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: consistency in message strings

[Derek Martin]

On Fri, Apr 24, 2020 at 09:57:59AM -0500, Derek Martin wrote:
> > Also note that year ranges (or list of years) are ambiguous. For
> > instance, this version of Mutt is not the one that was published
> > in 1996. The copyright notice should normally give only the year
> > of the first publication, which will be 2020 for version 1.14.
> 
> This is not completely accurate, at least under US law.  If you keep a
> mix of old and new content, your copyright date may be a range of
> years instead of a single year.  You will often see this in revised
> editions of old books, though the practice does seem to be less common
> recently.  This is the case with Mutt, and we had this conversation
> when Thomas revised the copyright notices the first time someone
> pointed out they weren't current while he was maintainer.

Some references:

Australia
  https://epiphany.law/articles/copyright/copyright-dates-single-year-or-range
International (only mentions websites and blogs explicitly)
  https://www.copyrightlaws.com/copyright-symbol-notice-year/ (International)
US
  
https://danashultz.com/2013/10/09/copyright-notice-with-multiple-years-legitimate/

I can't quickly find a specific reference to site, but my recollection
is that the idea here is that a frequently-updated work has multiple
years of "first publication", and expressing the date as a range of
years is how you cover that.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: consistency in message strings

[Derek Martin]

On Fri, Apr 24, 2020 at 04:16:00PM +0200, Vincent Lefevre wrote:
> On 2020-04-23 13:29:54 -0500, Derek Martin wrote:
> > On Mon, Apr 20, 2020 at 08:09:08PM +0200, ilf wrote:
> > > I think it's fair to honor Kevin as maintainer more than "others".
> > > I would propose something along: "Copyright (C) 1996-2016 Michael
> > > R. Elkins, 20XX to 2020 Kevin J. McCarthy, and others.
> > 
> > No, it MUST NOT say that.  That states that only Michael Elkins
> > contributed copyrightable work from 1996-2016, which is false, and
> > contradicts the previous copyright messages.
> > 
> > It should only update the date.  It would be acceptable to add names,
> > to the list, but... why?  If you do, be sure to add mine, OK? =8^) 
> > 
> > [This is why we do not do that.]
> 
> BTW, copyright notices do not say who contributed, but who owns
> the copyright. 

In most cases, this is a distinction without a difference,
particularly in Mutt.  The owner of a copyright is the person who
authored the work, i.e. the contributor, unless that person or entity
has legally assigned their copyright to someone else.  Mutt does not
ask its contributors to do this, so AFAIK the contributor is always
the copyright holder.

> In particular, even if Michael Elkins did not contributed after
> 2016, he is still part of the copyright owner in 2020, because the
> work (this version) still contains code he wrote (and this code has
> not fallen in the public domain).
> 
> Also note that year ranges (or list of years) are ambiguous. For
> instance, this version of Mutt is not the one that was published
> in 1996. The copyright notice should normally give only the year
> of the first publication, which will be 2020 for version 1.14.

This is not completely accurate, at least under US law.  If you keep a
mix of old and new content, your copyright date may be a range of
years instead of a single year.  You will often see this in revised
editions of old books, though the practice does seem to be less common
recently.  This is the case with Mutt, and we had this conversation
when Thomas revised the copyright notices the first time someone
pointed out they weren't current while he was maintainer.


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: consistency in message strings

[Derek Martin]

On Mon, Apr 20, 2020 at 08:09:08PM +0200, ilf wrote:
> Vincent Lefevre:
> > > > Copyright (C) 1996-2016 Michael R. Elkins and others.
> > > Thanks, but what should it be updated to say?  Michael Elkins hasn't
> > > contributed since 2016.  Is it correct to swap the date to 2020, or
> > > should I instead change the line to show my copyright?
> > Michael Elkins hasn't contributed since 2016, but "others" have.
> 
> I think it's fair to honor Kevin as maintainer more than "others". I would
> propose something along: "Copyright (C) 1996-2016 Michael R. Elkins, 20XX to
> 2020 Kevin J. McCarthy, and others.

No, it MUST NOT say that.  That states that only Michael Elkins
contributed copyrightable work from 1996-2016, which is false, and
contradicts the previous copyright messages.

It should only update the date.  It would be acceptable to add names,
to the list, but... why?  If you do, be sure to add mine, OK? =8^) 

[This is why we do not do that.]

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sun, Apr 19, 2020 at 07:38:39AM -0400, Remco Rijnders wrote:
> On Sat, Apr 18, 2020 at 09:13:34PM -0500, Derek wrote in
> <20200419021334.go19...@bladeshadow.org>:
> > OK I'm getting pretty bored with this, it's already been decided by
> > Kevin it won't be accepted, but I'll answer this last message since it
> > attempts to directly address a challenge I made.
> 
> Your initial response to my patch was in GB19544 from 19:59 on Friday. Then
> you moved on to do some other things and sent some other emails (on what and
> where I do not know, but I am reasonably confident there were two of them)
> before returning to this thread at 02:40 Friday on Saturday night in message
> GE19544. No further emails were sent by you till 12:04 on Saturday when you
> returned to this thread by message GF19544 -- I hope you slept well.
> 
> And then:
> GG19544   12:17
> GH19544   12:17
> GI19544   12:26
> GJ19544   12:33
> GK19544   13:12
> GL19544   13:23
> GM19544   18:23   (Did you go out to shop, or did you work a bit in the 
> garden?)
> GN19544   18:34
> GO19544   21:13

Are you kidding me with this?  As I've already pointed out, the times
I posted were already plainly visible, necessarily, in the headers of
my messages.  Needless to say I don't feel threatened by any of that,
and if I were inclined to be, I would not be posting messages in a
public forum.  The fact that I may have sent 2, or 28, or ... some
other number of messages which you can not in fact determine reliably
in between is not particularly interesting.  Presumably people use
their e-mail clients to send messages--that's what they're for.  You
learned, in sum total, exactly nothing.

What I came back to this thread to post about was that I'm glad to see
others discussing the difficulties with some of the suggestions, like
hashing (hash what? How do you make sure it remains unique/avoids
collisions, and is not susceptible to dictionary attack?), potentially
adding dependencies on cryptographic libraries, etc., so I don't have
to.  It's what I was hoping would happen, so I didn't have to be the
only one pointing that out.

This is actually a much harder problem than it seems like it should be
on the surface.  Given the general lack of sensitivity of the
information leaked, and how hard it is to actually produce a genuinely
unique message ID without leaking any info, I think it's really a
complete waste of time to spend any more time on this problem.  Mutt's
implementation is plenty good enough.  In the over 20 years I've been
on both Mutt lists, to my recollection I've never seen anyone post
about EITHER a legitimate attack vector related to this, OR a
practical problem caused by a real occurence of duplicated message
IDs--that, we would have taken pretty seriously.  It's been brought up
a few times, but IIRC purely in a theoretical context.  This problem
Just Doesn't Matter™.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

OK I'm getting pretty bored with this, it's already been decided by
Kevin it won't be accepted, but I'll answer this last message since it
attempts to directly address a challenge I made.

On Sat, Apr 18, 2020 at 08:00:24PM -0400, Remco Rijnders wrote:
> On Sat, Apr 18, 2020 at 06:23:53PM -0500, Derek wrote in
> <20200418232353.gm19...@bladeshadow.org>:
> > > i'm sure one could come up with other data points if one is inclined
> > > so.
> > 
> > This is not a compelling argument, and I'm equally sure that you can't.
> 
> "Ah... All this time you've been saying you do not know who Hot Lady 69 is,
> but I see that your and her mail come from the same process, on the same
> machine, with sequential ID's and just minutes apart. What gives?"
etc [...] 

None of these are also compelling, partly because they all describe
shady behavior, and I have never thought it to be Mutt's job to
protect you from your own shady behavior.  If you want to be shady, be
also not stupid, or be prepared to accept the consequences of being
caught being shady.

They also require either full access to all of your e-mail, or to a
very specific set of messages that bookends the period of bad
behavior.  That's not likely.

The Hot Lady example is also another example of what I described
before: it's irrelevant, because the message headers indicating the
message came from the same source are already enough to incriminate
you.  In the case of actual criminal behavior, it's likely enough to
justify a search warrant just about everywhere, if your local law
enforcement even requires such things...

The teen homework example is even more ridiculous because on top of
all of those things, it also requires that a teen be using mutt to
send e-mail, AND not be using instagram/snapchat instead, AND have a
parent who is watching them so closely AND understands how Mutt's
message IDs are generated AND IS ALSO NOT RUNNING THE E-MAIL SERVER
the teen is using, so they could just read the messages.  This is a
pretty outlandish set of unlikely circumstances.

None of this do I find compelling.

> If (as ilf pointed out but which you did not address in your response to
> him/her) the concerns raised in https://gitlab.com/muttmua/mutt/-/issues/159
> are valid there, why are they not valid in this context?

I don't see anything I wrote at that link, but since you asked, I'll
speak candidly.  I largely think that change was also fairly
pointless, however it's a change of default config only.  As such I
don't really care, anyone who wants to display their user agent still
can, and it was never a required part of the spec.  It does also
identify PRECISELY which version of Mutt you're running, unlike the
message-id generation, which may indicate very specifically what
attack vectors are available to attack you through Mutt.

I still don't consider that a very serious threat, because thankfully
there have been blessedly few serious security issues in Mutt over its
life.  I will say this is in at least some small part because I have
argued against code changes that provided little or no benefit or
which degraded security for 20+ years, and managed to stop more than a
few.

You're welcome. =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sun, Apr 19, 2020 at 01:03:06AM +0200, Oswald Buddenhagen wrote:
> On Sat, Apr 18, 2020 at 01:23:50PM -0500, Derek Martin wrote:
> > On Sat, Apr 18, 2020 at 01:57:50PM -0400, Remco Rijnders wrote:
> > > On Sat, Apr 18, 2020 at 12:26:34PM -0500, Derek wrote in
> > > > In other words, this scheme does not guarantee uniqueness, and is
> > > > therefore broken.
> > > 
> > > Well, the odds of the same number being selected are about 1 in 2 billion
> > > (on modern day Linux, admittedly), while the odds of winning the jackpot 
> > > in
> > > the Powerball lottery are about 1 in 292 million.
> > 
> > If your RNG is working properly.
> > 
> you have much bigger issues (with mutt alone) if it isn't.

Maybe... if for example you're reading your mail on a remote server
over an encrypted session (IMAP, etc.).  I don't.  Otherwise, off the
top of my head I think the biggest concern would be generating
temporary file names, but we could eliminate that as a concern also by
having mutt create a temporary directory to put the files in, and lock
it down to 700, prior to creating the files (in fact I think I've
suggested just that in the past).  An even better solution to that
problem is for the user to set TMPDIR to a local-disk directory
they've already created, that has restrictive permissions, and then
there would be no possibility of a collision when creating temporary
directories/files.  (I do this.)

After that, I don't think Mutt uses random numbers for anything
sensitive, that I can remember.  But obviously I'd have to review the
code.

> anyway, an alternative to randomizing would be sha1'ing everything except
> the timestamp - the probability of a clash is just beyond the pale. in fact,
> i think i've seen message ids that do just that, as they looked like
> .

I think this is a reasonable suggestion.  I still don't think it's
worth the effort.

> on a tangent, mutt's thread linking features do not work if the message-ids
> lack the . i presume these might be invalid, but they are
> rather common nonetheless. someone feels like having a look?

You should probably open a bug about it...

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sun, Apr 19, 2020 at 12:33:19AM +0200, Oswald Buddenhagen wrote:
> On Sat, Apr 18, 2020 at 12:04:05PM -0500, Derek Martin wrote:
> > OK, please enlighten me:  Tell me what you've learned,
> > 
> nothing, because i don't care. ;)
> 
> > how it's any worse than all the other information I demonstrated is
> > necessarily leaked from the headers, and how it is in any way
> > exploitable.
> > 
> as the initial mail indicated, this is about data-mining *habits*. i
> can use that to make a first guess about how insecure your system is
> (judging by a long uptime)

I've already explained why it's not a good guess.  And I've already
pointed out that even if you happened to guess correctly, it tells you
nothing about actual exploitable behavior.

There are things one could do that would result in always having the
same PID.  One is using $edit_headers to make it so.  A very
plausible, real-world example would be if I happen to boot my laptop
every day, and immediately start Mutt as the first thing I did, it
would very likely have the same pid, day after day, because the boot
sequence, and my behavior, are deterministic (at least until one of
those things changes).  That would look like a long uptime, but...not
be.

But it wouldn't matter, because this is not how someone with a clue
would look for ways to attack you.  And I'm not worried about someone
who doesn't have a clue.

> or make you feel paranoid by showing that i know how often you
> restart your MUA (who knows what _else_ i learned?).

I'm not interested in allaying uninformed paranoia. I'm only
interested in protecting Mutt's users from real threats, by plugging
legitimate holes.

> i'm sure one could come up with other data points if one is inclined
> so.

This is not a compelling argument, and I'm equally sure that you can't.
Since proving a negative is impossible, the burden is on you to prove
you can.  Do so, and I'll happily change my tune.  I can't know
everything so if you do find a way, I'll be wrong, and I won't be
embarrassed to admit it.  But I've been doing this kind of stuff for
quite a while, and I don't think your chances are very good.

> that my local host's name is revealed is mildly annoying, too. and
> yes, i could avoid revealing that anyway by having my MTA
> suppress/fake the Received headers appropriately for privacy (i
> didn't check what software/config i'd have to use, but i'm willing
> to bet that there _are_ options).

As I pointed out already, the first legitimate mail gateway you talk
to will add a header that includes the hostname or IP address of the
machine it was talking to, so there's really only one thing you can
do:  Use an anonymizer.  Most of those are pretty shady, and doing so
may be likely to attract the attention of law enforcement or
criminals.  Anyone who cares to will be able to determine that you're
using an anonymizer, which may suggest that you have something worth
hiding.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 01:57:50PM -0400, Remco Rijnders wrote:
> On Sat, Apr 18, 2020 at 12:26:34PM -0500, Derek wrote in
> > In other words, this scheme does not guarantee uniqueness, and is
> > therefore broken.
> 
> Well, the odds of the same number being selected are about 1 in 2 billion
> (on modern day Linux, admittedly), while the odds of winning the jackpot in
> the Powerball lottery are about 1 in 292 million.

If your RNG is working properly.

  https://www.schneier.com/blog/archives/2008/05/random_number_b.html

Mutt's message ID generation is not susceptible to such bugs.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

I'd like finally to elaborate on just a couple more points:

On Sat, Apr 18, 2020 at 10:47:58AM -0700, Kevin J. McCarthy wrote:
> On Sat, Apr 18, 2020 at 08:41:47AM -0400, Remco Rijnders wrote:
> > On Fri, Apr 17, 2020 at 07:14:02PM -0700, Kevin wrote:
> > > I would also note that the current message id generation algorithm
> > > is deterministic.  It's easy to understand how it generates, and to
> > > know the exact limitations where it might fail.  Swapping out those
> > > bits of uniqueness with a random() call takes away the determinism.
> > > 
> > > The tiny (in my opinion too) information leaks, are not worth that
> > > trade-off.
> > 
> > I think having the message ID being non deterministic would be of
> > greater value. The more deterministic it is, the easier it would be for
> > someone to create a clash.
> 
> The opposite, I would say.  Unless you mean adversarially.  But there is no
> such threat model: I can easily steal your message id after you've sent the
> email.

Note that what is deterministic is its uniqueness.  Of course, if you
want to abuse the mail system, that's easy... you can tell Mutt to
always use the same message ID, for example.  The main consequence of
this is that everyone you send mail to will see all your messages as
being the same message/thread, so it will give them headaches.  But
you can do it, and the consequences are pretty minor.  But Mutt wants
to be a good citizen of the e-mail community, so it will try hard to
meet the requirements established by that community, to prevent this
kind of nonsense.

> > I can't think of an example why someone would want to do that, but I
> > also can't really think of an example where a (fully) deterministic
> > Message-ID would be of value.

I think I just explained that.  Message IDs overlapping breaks
threading.  It also breaks tracing and debugging via logs, etc..

> The date/time partitions by time.  The host by... host.  The pid by mutt
> instance.  The counter within a single instance.
> 
> With this, I can say that two mutt instances on the same machine won't
> generate the same message id, even if they are sent within the same second.
> The same mutt instance also won't do that, unless you send >26 messages in a
> single second.

And let's be clear about what it would take to generate messages with
the same ID in Mutt:

 - You need to generate > 26 messages in the same second--not any one
   second, but XX:XX:XX.00 - XX:XX:XX.99.  This is actually very
   difficult to do.
+ You'd need to be watching the system clock to wait for XX.00
+ Messages take time to be generated
+ Messages take time to be submitted to SMTP
+ A DNS lookup may be required, depending on settings
 - You need to do it from the same pid
 - You need to do it on the same machine

> Is this perfect?  No.  But your patch can no longer say even say that. It
> can merely say "probably".

It's pretty close to perfect.  In order to generate the necessary
number of messages in the requisite 1s window, the easiest way would
be to write a script to generate 26 separate processes simultaneously,
but then each one would have a different PID field in their message
ID.  To do it from the same process, you'd need a very fast machine,
a persistent connection to the SMTP gateway, and a very complicated
macro to create the messages as fast as possible.

In other words, the only way you're going to break Mutt's message ID
uniqueness is if you're actually trying to, really, really hard, and
probably not even then.

If you actually were trying to, it would be much easier to just use
$edit_headers and set it to whatever you want.  Which is exactly what
you should do, if you care about this information leak.

> If there were a *real* threat model, Derek and I would take this more
> seriously.  But I'm not going to backtrack on the generator determinism just
> to satisfy vague "security" threats.

Indeed.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 08:27:55PM +0300, Consus wrote:
> > > Nope, hostname it is, even if you're using built-in smtp.
> > 
> > False.  Try it. 
> 
> Check my headers :)
> 
> > Tell mutt your hostname is your domain name.  This is a common
> > configuration, so that mutt auto-generates your e-mail address as
> > myn...@mydomain.org instead of myn...@myhost.myname.org.  If you do
> > that it will generate a message ID that is your domain, not your
> > hostname.  I actually tested this myself rather than blindly asserting
> > it.
> 
> That requires _explicit_ configuration. So it's not that usual, not many
> people configure 'hostname', most of the time people just set 'from'.

I'm not sure what gives you that idea.  It used to be something that
we gave as general advice for configuring mutt, and I personally
fixed a bug in Mutt that could result in the hostname being set to an
invalid value if it was not explicitly set, suggesting that rather a
lot of people were setting it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 06:14:58PM +0200, ilf wrote:
> IMHO all the arguments also apply here. IMHO we should chose a form of
> Message-ID that (a) does not include unneccessary system information and (b)
> matches that of other MUAs. Maybe we should follow these "Recommendations
> for generating Message IDs"?
> https://tools.ietf.org/html/draft-ietf-usefor-message-id-01

Using a standard method for generating IDs is one I would support in
general, as I believe in standards conformance in absentia of a good
reason not to conform.  Gowever in this case, it appears the
referenced document never made it past draft status.  It could be
because the document itself describes its own flaw, which is the same
flaw I have just explained in another message that the one being
discussed here has:

3.2.2. Using a psuedorandom number generator

One could take 64 bits from a good, well-seeded pseudorandom number
generator [PRNG] in order to significantly increase the uniqueness of
the Message-ID.  The advantage of this method is that it is fast and
generally effective.  The disadvantage is that in a perfect random
number generation scheme, the possibility of getting the same number
twice in a row is exactly the same probability as getting any two
numbers.

In other words, this scheme does not guarantee uniqueness, and is
therefore broken.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 01:10:47PM +0300, Consus wrote:
> On Fri, Apr 17, 2020 at 07:59:01PM -0500, Derek Martin wrote:
> > On Fri, Apr 17, 2020 at 02:24:22PM -0400, Remco Rijnders wrote:
> > > The Message-ID that mutt generates is supposed to be unique. Up till now
> > > mutt would generate this ID based on the current date and time, followed 
> > > by
> > > ".G". followed by a letter A to Z (A for the 1st and 27th email sent, Z 
> > > for
> > > the 26th, etc.), followed by the pid of the active mutt process, followed
> > > by "@" and the configured hostname.
> > 
> > This is utterly pointless.  This may come off as harsh but please
> > understand that's not intended.  I just want to be completely clear
> > hee so there is no misunderstanding or equivocation.
> > 
> > None of the information you just listed is sensitive, and almost all
> > of it is already REQUIRED to be present in the message:
> >
> >  - The "hostname" is usually the sender's domain, not their actual
> >hostname, unless left unconfigured in Mutt.  Regardless of which
> >thing it is, it's going to be all over the message headers for the
> >vast majority of Mutt users.  In those cases when it won't, the
> >user's IP address will be in them at least once (and might be
> >anyway, depending on how the user emits mail into the SMTP ether
> >and who it is talking to). REQUIRED.
> 
> Nope, hostname it is, even if you're using built-in smtp.

False.  Try it.  Tell mutt your hostname is your domain name.  This is
a common configuration, so that mutt auto-generates your e-mail
address as myn...@mydomain.org instead of myn...@myhost.myname.org.
If you do that it will generate a message ID that is your domain, not
your hostname.  I actually tested this myself rather than blindly
asserting it.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 08:26:56AM -0400, Remco Rijnders wrote:
> > - the PID is the only thing that could possibly be vaguely useful to
> >   an attacker, but only if they're already able to get onto the
> >   user's system, in which case finding out the PID will be trivial
> >   anyway. POINTLESS.
> 
> I would argue including the PID in the message ID is equally or even more
> pointless. It has no value in there, so why even include it?

Because it deterministically provides uniqueness.  At the time it was
created, it is not possible for there to be two processes with that
PID.

> > - From the sequential letter portion, you can only determine that the
> >   modulo 26 of the number of messages sent, not the number of
> >   messages.  That's not useful information for anything, and I doubt
> >   the actual number of messages sent in a given mutt session reveals
> >   anything useful either, even if it were available--you still have
> >   no idea if the session has been running for 10 minutes or 10 years.
> >   MEANINGLESS.
> 
> Also meaningless to include this then.

Probably, but it provides additional uniqueness.  It's extremely
unlikely you'll be able to create more than 26 messages in one second.

MOST IMPORTANTLY: note that by replacing this info with a randomly
generated number, you are REMOVING the guarantee of uniqueness.  While
it is IMPROBABLE that the randomly generated number will repeat, it is
not impossible.  The existing method does not have this flaw.

> > I haven't reviewed the patch, but it does nothing useful, so my main
> > objection is that taking the time to review it, let alone apply it, is
> > a waste of anyone's time.
> 
> I think a lot more time was wasted not looking at the patch and writing your
> reply than having had a quick glance at it.

It was not wasted if anyone learned anything.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 11:07:34AM +0200, Oswald Buddenhagen wrote:
> > Because it adds additional complexity to the program for absolutely zero
> > practical gain.
> > 
> actually, the patch makes the code simpler.

I admitted already I did not look at the patch, so I was speaking in
general terms here, obviously.

> > Because it would require non-zero work from maintainers to review,
> > apply, and commit the change, for absolutely zero benefit.
> > 
> that's silly. all of that effort would have been less than what you already
> spent on arguing against that patch.

Again, speaking in general terms.  However, time spent educating
someone has value.  Time spent on code changes that don't actually
improve the code has none.

> > On Sat, Apr 18, 2020 at 04:17:45AM +0200, Gero Treuner wrote:
> > > With medium probability it could indicate whether it is among the
> > > first 26 messages of a session.
> > 
> > It absolutely can not.  If you sent 1 message, the letter would be A.
> > If you sent 27 messages, the letter would be A. [...]
> > 
> yeah, thanks for explaining some basic algebra, but you're still
> wrong.  gero made a point about probability.  and by analysing
> multiple messages from you, one can identify patterns. especially in
> conjunction with the oh-so-meaningless pid.

OK, please enlighten me:  Tell me what you've learned, how it's any
worse than all the other information I demonstrated is necessarily
leaked from the headers, and how it is in any way exploitable.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Sat, Apr 18, 2020 at 04:17:45AM +0200, Gero Treuner wrote:
> On Fri, Apr 17, 2020 at 07:59:01PM -0500, Derek Martin wrote:
> > On Fri, Apr 17, 2020 at 02:24:22PM -0400, Remco Rijnders wrote:
> > > The Message-ID that mutt generates is supposed to be unique. Up till now
> > > mutt would generate this ID based on the current date and time, followed 
> > > by
[...]
> But if there are people who care ... why not.

Because those people don't know what they're talking about, and
humoring them helps no one.  Because it adds additional complexity to the
program for absolutely zero practical gain.  Because it would require
non-zero work from maintainers to review, apply, and commit the
change, for absolutely zero benefit.

> There is a little room for improvement.

Sorry, no, there just isn't.  You can use any scheme you like, but it
will not leak any less information THAT MATTERS than Mutt's current
one does, no matter what it is, because the amount of information that
Mutt's scheme leaks THAT MATTERS is zero.  I'll demonstrate...

> > None of the information you just listed is sensitive, and almost all
> > of it is already REQUIRED to be present in the message:
> > 
> >  - The "hostname" is usually the sender's domain, not their actual
> >hostname
[...]
> I found that if I sent with From: of another domain that it didn't
> affect the MessageId, so indeed leaking some small part of information.

Sorry, you don't seem to understand how SMTP works.

Your message came from:

Received: from faun.innocircle.com (faun.innocircle.com [178.63.104.83])
by hemlock.osuosl.org (Postfix) with ESMTPS id 677388828E
for ; Sat, 18 Apr 2020 02:54:26 + (UTC)
Received: from local by faun.innocircle.com (envelope-from 
)
id 1jPd3B-0004ds-7z; Sat, 18 Apr 2020 04:17:45 +0200

I changed your e-mail address because while the above headers are not
normally available in mailing list archives' web interface (anymore),
and they also modify e-mail addresses in the envelope headers to help
keep spam away (although it's a very simple transformation which is
very easily undone), the message bodies are stored unmodified,
including any message headers that are in them.  Anyway...

These headers are added to your message by your machine, your SMTP
gateway, and every mail forwarder in between you and me.  So including
it in the message id is only leaking info that is, as I said,
REQUIRED, and therefore already present in the message.  I know not
only the date and time, your host name and IP address, time zone, MTA
software, but also the exact route your message took to get to me, and
with high probability the city you live in--or at least where your ISP
or employer does.  I won't post it here unless you want me to prove
that.

There ARE things you can do to obscure these headers...  None of these
are things which would be done by well-behaved legitimate mail
servers, nor are they things the average mail user, or the average
Mutt user--or even most exceptional mutt users--will do to prevent
their machine name and IP from being discovered.

> >  - the PID is the only thing that could possibly be vaguely useful
> >  to an attacker, but only if they're already able to get onto the
> >user's system, in which case finding out the PID will be trivial
> >anyway. POINTLESS.
> 
> With small probability it could indicate whether your system was
> recently started or Mutt is usually started right after system start.

It absolutely can not. PIDs are limited, and while modern systems
COULD use 64-bit PIDs, in practice on most systems you only get 32K of
them, and then they start wrapping around.  On a busy system, that's
not really a lot of pids.  Knowing the PID tells you ABSOLUTELY NOTHING.

But even if it did, it doesn't really present any sort of attack
vector, as I said, unless the attacker is already on the same
machine... and then it doesn't matter anyway.  And if it did somehow
matter, then every Unix system in existence would be vulnerable,
because you can find out PIDs with a simple command that anyone on the
machine can run.

> >  - From the sequential letter portion, you can only determine that the
> >modulo 26 of the number of messages sent, not the number of
> >messages.  That's not useful information for anything, and I doubt
> >the actual number of messages sent in a given mutt session reveals
> >anything useful either, even if it were available--you still have
> >no idea if the session has been running for 10 minutes or 10 years.
> >MEANINGLESS.
> 
> With medium probability it could indicate whether it is among the first
> 26 messages of a session.

It absolutely can not.  If you sent 1 message, the letter would be A.
If you sent 27 messages, the letter would be A.  131 messages, the
letter would be A.

Re: [PATCH] Change Message-ID generation to be more unique and leak less information

[Derek Martin]

On Fri, Apr 17, 2020 at 02:24:22PM -0400, Remco Rijnders wrote:
> The Message-ID that mutt generates is supposed to be unique. Up till now
> mutt would generate this ID based on the current date and time, followed by
> ".G". followed by a letter A to Z (A for the 1st and 27th email sent, Z for
> the 26th, etc.), followed by the pid of the active mutt process, followed
> by "@" and the configured hostname.

This is utterly pointless.  This may come off as harsh but please
understand that's not intended.  I just want to be completely clear
hee so there is no misunderstanding or equivocation.

None of the information you just listed is sensitive, and almost all
of it is already REQUIRED to be present in the message:

 - The date and time will be in at least one, probably multiple
   headers, guaranteed; and quite possibly the message body, depending
   on the user's habits.  REQUIRED.
   
 - The "hostname" is usually the sender's domain, not their actual
   hostname, unless left unconfigured in Mutt.  Regardless of which
   thing it is, it's going to be all over the message headers for the
   vast majority of Mutt users.  In those cases when it won't, the
   user's IP address will be in them at least once (and might be
   anyway, depending on how the user emits mail into the SMTP ether
   and who it is talking to). REQUIRED.

 - the PID is the only thing that could possibly be vaguely useful to
   an attacker, but only if they're already able to get onto the
   user's system, in which case finding out the PID will be trivial
   anyway. POINTLESS.

 - From the sequential letter portion, you can only determine that the
   modulo 26 of the number of messages sent, not the number of
   messages.  That's not useful information for anything, and I doubt
   the actual number of messages sent in a given mutt session reveals
   anything useful either, even if it were available--you still have
   no idea if the session has been running for 10 minutes or 10 years.
   MEANINGLESS.
 
 - the rest is arbitrary noise. MEANINGLESS.

Besides all that, Mutt already provides a means of you to eliminate
any information leak in the message ID via $edit_headers, but again,
making use of that facility for that purpose is pointless.

I haven't reviewed the patch, but it does nothing useful, so my main
objection is that taking the time to review it, let alone apply it, is
a waste of anyone's time.

And yes, we've had this conversation before.  In 2001:

  https://marc.info/?l=mutt-dev=100428813825414=2


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Limit beeping mailboxes

[Derek Martin]

On Tue, Mar 10, 2020 at 12:43:54PM +0300, Consus wrote:
> Hi,
> 
> Have you considered ability to limit mailboxes that "beep" on new mail?
> Say, I have a bunch of IMAP mailboxes:

I think you can get what you want if you don't subscribe (i.e. use the
"mailboxes" command) to mailboxes you don't want new mail notification
for.  IIRC, you can still navigate to them through the browser. [You
definitely can for local mailboxes, and I think you can for IMAP as
well, but it's been a long, long while since I used IMAP.] 


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Attachments counter broken?

[Derek Martin]

On Tue, Mar 03, 2020 at 10:07:04AM -0800, Kevin J. McCarthy wrote:
> On Tue, Mar 03, 2020 at 10:40:52AM +0300, Consus wrote:
> > It seems like attachments counter (%X) is broken, at least in
> > pager_format. It always expands to 0, no matter how many attached files
> > there is.
> 
> It seems to be working for me.
> 
> You'll need to add 'attachments' commands to tell Mutt what to match and
> count as an attachment.  See
> 

Does it make sense that the user should need to configure this?  It
seems like the Mutt devs would have a much better idea what
constitutes an attachment (from a technical standpoint) than the
average user...

Personally, I don't care about this feature TBH, but if I did, the way
I would expect that it worked was that Mutt provided sane defaults
(perhaps compiled in, or at least part of the system Muttrc), and
optionally, users could override those if they found it necessary.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Planning stable 1.12.2 release this weekend

[Derek Martin]

On Mon, Sep 16, 2019 at 11:16:43AM -0700, Kevin J. McCarthy wrote:
> Just a heads up that I plan on releasing 1.12.2 this coming weekend.
> 
> Derek, I feel confident in Gero's fix, but if you have the chance to compile
> and test before the weekend that would be great.

Tried it, looks good!  And disabling gpg agent also was the fix for my
signing issue.

Thanks very much!

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



signature.asc
Description: PGP signature

Re: Planning stable 1.12.2 release this weekend

[Derek Martin]

On Mon, Sep 16, 2019 at 11:16:43AM -0700, Kevin J. McCarthy wrote:
> Just a heads up that I plan on releasing 1.12.2 this coming weekend.
> 
> Derek, I feel confident in Gero's fix, but if you have the chance to compile
> and test before the weekend that would be great.

I'm still trying.  I've just been swamped.  And sick.  Probably from
drinking too much swamp water... =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

Re: make fails

[Derek Martin]

On Mon, Sep 09, 2019 at 06:12:49PM -0700, Kevin J. McCarthy wrote:
> On Wed, Sep 04, 2019 at 10:26:31AM -0500, Derek Martin wrote:
> > I'll try to look at this no later than this weekend...
> 
> Derek, I've merged Gero's patch into the stable branch in git.  It should
> compile with inotify support for you now.  If you have the chance, please
> try compiling the stable git branch instead.
> 
> Thank you!

Thanks Kevin, I never did get the chance to look into this, but I will
hopefully have time tonight.


-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

Re: make fails

[Derek Martin]

I'll try to look at this no later than this weekend...

Thanks!

On Sun, Sep 01, 2019 at 06:19:17PM +0200, Gero Treuner wrote:
> Hi Derek,
> 
> On Wed, Aug 28, 2019 at 02:16:10PM -0700, Kevin J. McCarthy wrote:
> > On Mon, Aug 26, 2019 at 02:12:56PM -0500, Derek Martin wrote:
> > > /home/ddm/tmp/mutt-1.12.1/buffy.c:276: undefined reference to 
> > > `mutt_monitor_remove'
> > > /home/ddm/tmp/mutt-1.12.1/buffy.c:310: undefined reference to 
> > > `mutt_monitor_remove'
> > > /home/ddm/tmp/mutt-1.12.1/buffy.c:326: undefined reference to 
> > > `mutt_monitor_add'
> > 
> > I think I know what the problem is: it's an improper usage of
> > AC_CHECK_FUNCS.  I'm guessing your system has one or more of
> > "inotify_init1() inotify_add_watch() inotify_rm_watch()" but not all of
> > them.
> 
> Probably only inotify_init1() is missing, which can easily be replaced.
> 
> If it's this - verifyable by config.status - then the attached patch
> helps not only to compile, but makes inotify running for Mutt on your
> system.  I tested it by simulating this scenario in the build, leading
> to a binary using the alternate function.
> 
> > I'll push a patch to stable, but for now just keep the
> > '--disable-filemontor'.  The fix will just end up disabling it on your
> > system.
> 
> I created a merge request in GitLab. 
> 
> If Derek can confirm that we are guessing right (I noticed that this
> might not be soon) this can be committed.
> 
> 
> Gero

> diff -ur mutt-1.12.1.orig/config.h.in mutt-1.12.1/config.h.in
> --- mutt-1.12.1.orig/config.h.in  2019-06-15 18:07:25.0 +0200
> +++ mutt-1.12.1/config.h.in   2019-09-01 17:38:05.794709214 +0200
> @@ -223,6 +223,9 @@
>  /* Define to 1 if you have the `inotify_add_watch' function. */
>  #undef HAVE_INOTIFY_ADD_WATCH
>  
> +/* Define to 1 if you have the `inotify_init' function. */
> +#undef HAVE_INOTIFY_INIT
> +
>  /* Define to 1 if you have the `inotify_init1' function. */
>  #undef HAVE_INOTIFY_INIT1
>  
> diff -ur mutt-1.12.1.orig/configure mutt-1.12.1/configure
> --- mutt-1.12.1.orig/configure2019-06-15 18:04:19.0 +0200
> +++ mutt-1.12.1/configure 2019-09-01 17:38:05.670708221 +0200
> @@ -619,6 +619,7 @@
>  # include 
>  #endif"
>  
> +ac_func_list=
>  ac_header_list=
>  ac_subst_vars='am__EXEEXT_FALSE
>  am__EXEEXT_TRUE
> @@ -2610,6 +2611,7 @@
>>$cache_file
>  fi
>  
> +as_fn_append ac_func_list " inotify_init1"
>  as_fn_append ac_header_list " stdlib.h"
>  as_fn_append ac_header_list " unistd.h"
>  as_fn_append ac_header_list " sys/param.h"
> @@ -11065,7 +11067,7 @@
>  
>  
>  if test x$have_filemonitor != xno ; then
> -for ac_func in inotify_init1 inotify_add_watch inotify_rm_watch
> +for ac_func in inotify_init inotify_add_watch inotify_rm_watch
>  do :
>as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
>  ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
> @@ -11074,18 +11076,33 @@
>  #define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1
>  _ACEOF
>  
> +else
> +  have_filemonitor=no
> +fi
> +done
> +
> +if test x$have_filemonitor != xno ; then
>  
>  $as_echo "#define USE_INOTIFY 1" >>confdefs.h
>  
>  
> -else
>  
> -have_filemonitor=no
> +
> +  for ac_func in $ac_func_list
> +do :
> +  as_ac_var=`$as_echo "ac_cv_func_$ac_func" | $as_tr_sh`
> +ac_fn_c_check_func "$LINENO" "$ac_func" "$as_ac_var"
> +if eval test \"x\$"$as_ac_var"\" = x"yes"; then :
> +  cat >>confdefs.h <<_ACEOF
> +#define `$as_echo "HAVE_$ac_func" | $as_tr_cpp` 1
> +_ACEOF
>  
>  fi
>  done
>  
> -if test x$have_filemonitor != xno ; then
> +
> +
> +
>  for ac_header in sys/inotify.h
>  do :
>ac_fn_c_check_header_mongrel "$LINENO" "sys/inotify.h" 
> "ac_cv_header_sys_inotify_h" "$ac_includes_default"
> diff -ur mutt-1.12.1.orig/configure.ac mutt-1.12.1/configure.ac
> --- mutt-1.12.1.orig/configure.ac 2019-06-15 17:57:01.0 +0200
> +++ mutt-1.12.1/configure.ac  2019-09-01 17:27:18.933520732 +0200
> @@ -854,14 +854,11 @@
>  ])
>  
>  if test x$have_filemonitor != xno ; then
> -AC_CHECK_FUNCS(inotify_init1 inotify_add_watch inotify_rm_watch,
> -[
> -AC_DEFINE(USE_INOTIFY,1,[ Define if want to use inotify for 
> filesystem monitoring (available in Linux only). ])
> -],

Re: make fails

[Derek Martin]

On Wed, Aug 28, 2019 at 02:19:39PM -0700, Kevin J. McCarthy wrote:
> On Mon, Aug 26, 2019 at 02:29:22PM -0500, Derek Martin wrote:
> > However, just now, trying to pgp-sign this message, I got:
> > 
> > gpg: can't query passphrase in batch mode
> > gpg: skipped "0xDFBEAD02": bad passphrase
> > gpg: signing failed: bad passphrase
> 
> Try adding 'unset pgp_use_gpg_agent' to your muttrc.  Starting in 1.12, I
> defaulted that on because of the increasing prevalence of gpg 2.1 (which
> uses an agent by default).
> 
> Also, make sure to use the latest contrib/gpg.rc settings.  Some of them are
> now necessary for proper decryption and signing verification.
> 
Thanks Kevin--I still haven't had any spare cycles to look at this,
but when I do this will be the first thing I look at. =8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

Re: make fails

[Derek Martin]

On Mon, Aug 26, 2019 at 02:12:56PM -0500, Derek Martin wrote:
> I may also just configure with --disable-filemonitor and try again,
> but I suspect I want whatever awesomeness that brings (I vaguely
> remember a discussion about this but sadly not the details).

FWIW this worked.

However, just now, trying to pgp-sign this message, I got:

gpg: can't query passphrase in batch mode
gpg: skipped "0xDFBEAD02": bad passphrase
gpg: signing failed: bad passphrase
Press any key to continue...

But I'm not in batch mode. :-(

I had to remove PGP options in order to send this. :-(

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.

make fails

[Derek Martin]

Folks,

I downloaded mutt-1.12.1 today, and tried to compile it.  I ran
configure thusly:

./configure --enable-sidebar --enable-compressed  --enable-hcache

Then make failed to link, thusly:

gcc -std=gnu99  -Wall -pedantic -Wno-long-long -g -O2  -L/lib -o mutt 
addrbook.o alias.o attach.o base64.o browser.o buffer.o buffy.o color.o crypt.o 
cryptglue.o commands.o complete.o compose.o copy.o curs_lib.o curs_main.o 
date.o edit.o enter.o flags.o init.o filter.o from.o getdomain.o group.o 
handler.o hash.o hdrline.o headers.o help.o hook.o keymap.o main.o mbox.o 
menu.o mh.o mx.o pager.o parse.o pattern.o postpone.o query.o recvattach.o 
recvcmd.o rfc822.o rfc1524.o rfc2047.o rfc2231.o rfc3676.o score.o send.o 
sendlib.o signal.o sort.o status.o system.o thread.o charset.o history.o lib.o 
muttlib.o editmsg.o mbyte.o url.o ascii.o crypt-mod.o safe_asprintf.o 
patchlist.o conststrings.o  pgp.o pgpinvoke.o pgpkey.o pgplib.o gnupgparse.o 
pgpmicalg.o pgppacket.o crypt-mod-pgp-classic.o smime.o 
crypt-mod-smime-classic.o sidebar.o compress.o resize.o hcache.o md5.o 
mutt_idna.o   -lncursesw -lidn   -L/usr/lib// -ldb-4.3
buffy.o: In function `mutt_parse_mailboxes':
/home/ddm/tmp/mutt-1.12.1/buffy.c:276: undefined reference to 
`mutt_monitor_remove'
/home/ddm/tmp/mutt-1.12.1/buffy.c:310: undefined reference to 
`mutt_monitor_remove'
/home/ddm/tmp/mutt-1.12.1/buffy.c:326: undefined reference to `mutt_monitor_add'
curs_lib.o: In function `mutt_monitor_getch':
/home/ddm/tmp/mutt-1.12.1/curs_lib.c:141: undefined reference to 
`mutt_monitor_poll'
curs_main.o: In function `mutt_index_menu':
/home/ddm/tmp/mutt-1.12.1/curs_main.c:636: undefined reference to 
`mutt_monitor_add'
/home/ddm/tmp/mutt-1.12.1/curs_main.c:1342: undefined reference to 
`mutt_monitor_remove'
/home/ddm/tmp/mutt-1.12.1/curs_main.c:1390: undefined reference to 
`mutt_monitor_add'
/home/ddm/tmp/mutt-1.12.1/curs_main.c:1356: undefined reference to 
`mutt_monitor_add'
collect2: ld returned 1 exit status
make[2]: *** [mutt] Error 1
make[2]: Leaving directory `/home/ddm/tmp/mutt-1.12.1'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/ddm/tmp/mutt-1.12.1'
make: *** [all] Error 2

I have made no effort to determine why, as of yet.  I don't have the
cycles right now.  I may try after work, if I'm not exhausted,
assuming I don't see a reply that explains it before then.  I may also
just configure with --disable-filemonitor and try again, but I suspect
I want whatever awesomeness that brings (I vaguely remember a
discussion about this but sadly not the details).

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgp0t2yiyPukp.pgp
Description: PGP signature