Re: message-Id has localhost
Ken Hornstein wrote: > To Mike's question: >> Can we just use "localname" from mts.conf? > We COULD, it would just be wrong for some people. Agreed, but it was the first place that I looked to fix things. > That's the "local" hostname, and is used for a bunch of things INCLUDING > constructing the default hostname for email addresses. But here's a > thought experiment: let's say you set it to 'gmail.com' because your > email is hosted at gmail. There's no way you could guarantee your > Message-ID isn't going to be used by gmail.com already. Yes, you could I think that's reasonable, but is there actually anything anyone could do here? They really need to pick something else which is not gmail.com. > things. We could add another knob, but honestly I'd rather people just > use 'random' if the existing logic doesn't work for you. That's fair enough. I can live with random as well. > Well, technically, it's constructing the Message-ID based on the value > of the 'j' Sendmail macro, which is used for a ton of things; that macro > value is configurable and in my limited sendmail experience you usually > do explicitly configure it (I do not know what that defaults to). Yes, exactly, I was going to quibble at your analysis, but it wasn't worth it. All I'm saying is that gethostname() is probably better default than whatever it was that you posted that canonicalized it. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works|IoT architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ signature.asc Description: PGP signature
Re: message-Id has localhost
Hi Andy, > I also find it hard to beleive that someone wants the MUA to have a > specific Message-ID for their email People do all kinds of odd things. Like fully justify text on a media with coarse adjustment. :-) Wanting to specify the whole message-ID field is distinct from wanting to set the trailing ‘@...’ part. - The whole field could just be given as yet another field in the header, though currently nmh will complain it's an ‘illegal header line’. - Setting just the domain part is a more common need. It's tied in to adjusting the From field, and which party takes on the email for delivery. Otherwise, emails from foo.com leak the bar.com identity. -- Cheers, Ralph.
Re: message-Id has localhost
Hi Ken, > I do not believe there is solution to this that will universally work, > or even work in a large majority of cases considering the > configuration of the modern Internet. Agreed. That's why I suggested allowing ‘@...’ in addition to ‘local’ and ‘random’. It allows the user to route around the imperfect heuristics. -- Cheers, Ralph.
Re: message-Id has localhost
tl;dr: old man grouses at clouds On Sun, Dec 31, 2023 at 11:57 AM Ken Hornstein wrote: > [...] Sendmail, > yes, it looks like you could change it if you really want to; it also > defaults to something based on the local hostname. I am personally > skeptical that people actually configure this. > FWIW, MIT's campus computer network (Athena) did this for a long time, because that network was composed of thousands of workstations that did not normally receive mail and all wanted to send mail that came from, for example, rather than something like < yand...@w20-575-77.mit.edu>. I imagine that they stopped doing this when MIT moved to centralized (Office 365-based, gack) mail infrastructure rather than running their own (with the attendant large reductions in overhead, functionality, stability, etc.) That said, this is probably the exception that proves the rule, as keeping the sendmail.cf changes up to date was a constant sink for work even before the advent of dkim/dmarc/spf/eieio. ~Chad
Re: message-Id has localhost
>I also find it hard to beleive that someone wants the MUA to have a >specific Message-ID for their email, but there is at least one software >that I'm aware of that does act upon the contents of it: > >http://smarden.org/qconfirm/qconfirm-check-mid.1.html I mean, yes, it looks for messages based on Message-IDs in the References header; I think it would work fine with 'random'. >> My personal feeling is that the people who (a) care about generating a >> local Message-ID, and (b) actually care WHAT appears right of the '@' >> either need to configure their system appropriately or write code to >> change nmh behavior. > >I think that's reasonable. I don't see anything necessarily wrong with >nmh being able to generate a Message-ID so I'm sure patches would be >considered if someone offered. Just so I'm clear, I'm fine with people submitting patches to change the current behavior. --Ken
Re: message-Id has localhost
Thus said Ken Hornstein on Sun, 31 Dec 2023 15:00:38 -0500: > >Can we just use "localname" from mts.conf? > > We COULD, it would just be wrong for some people. I definitely don't want nmh generating Message-ID unless I ask it to do so as my MTA is already correctly configured. The MTA that I use, qmail, does not use the hostname, it uses one of 2 configuration files... /var/qmail/control/me by default unless /var/qmail/control/idhost exists in which case it uses that. However, that only allows me to customize the right-side of the Message-ID. The left-side is predetermined and uses basically what the RFC recommends (a timestamp and a PID). Andy
Re: message-Id has localhost
Thus said Ken Hornstein on Sun, 31 Dec 2023 11:56:38 -0500: > I am personally skeptical that people actually configure this. I also find it hard to beleive that someone wants the MUA to have a specific Message-ID for their email, but there is at least one software that I'm aware of that does act upon the contents of it: http://smarden.org/qconfirm/qconfirm-check-mid.1.html It seems reasonable to believe that I may want control over the Message-ID for the purpose of making this work, however, it's definitely not required that the MUA be responsible for this. > My personal feeling is that the people who (a) care about generating a > local Message-ID, and (b) actually care WHAT appears right of the '@' > either need to configure their system appropriately or write code to > change nmh behavior. I think that's reasonable. I don't see anything necessarily wrong with nmh being able to generate a Message-ID so I'm sure patches would be considered if someone offered. Andy
Re: message-Id has localhost
>> [...] Sendmail, >> yes, it looks like you could change it if you really want to; it also >> defaults to something based on the local hostname. I am personally >> skeptical that people actually configure this. >> > >FWIW, MIT's campus computer network (Athena) did this for a long time, >because that network was composed of thousands of workstations that did not >normally receive mail and all wanted to send mail that came from, for >example, rather than something like < >yand...@w20-575-77.mit.edu>. What you're talking about is a very common way of configuring Sendmail and I've personally done that many times; I call that a "site" configuration where all email that is submitted to the main Sendmail server with an internal hostname (or no hostname) is re-written to have the 'site' domain name. But what I was specifically talking about was that I am skeptical that anyone specifically configures a Message-ID header to be added by sendmail that is different than the default, which is based on the 'j' macro. I just looked in my wife's Sendmail 'bat' book and it says j holds the FQDN of the local machine, which probably means it does something similar to what nmh does; you can override that value if Sendmail gets it wrong, _HOWEVER_ it's used by a bunch of things and not just for Message-ID generation. So to one of Ralph's earlier points, it seems like we are using some MTA prior art, it just that it doesn't work for everybody. --Ken
Re: message-Id has localhost
>> I mean, that's not a reason in my thinking? Like, WHY do people want >> that? > >To be able to uniquely refer to that email in future by knowing what the >message-id field contains. The reference may be to oneself or to other >recipients. That is the purpose of the field. Not knowing the field's >value lessens its worth to tracing the flow through downstream parties >in log files. I think we're probably not going to agree whether or not those reasons qualify as "vague", but, fine. That's not really my point; I honestly don't care if people want to generate their own Message-IDs. What I _do_ care about is when they do and then complain that nmh is using the "wrong" hostname to do so; I do not believe there is solution to this that will universally work, or even work in a large majority of cases considering the configuration of the modern Internet. To Mike's question: >Can we just use "localname" from mts.conf? We COULD, it would just be wrong for some people. That's the "local" hostname, and is used for a bunch of things INCLUDING constructing the default hostname for email addresses. But here's a thought experiment: let's say you set it to 'gmail.com' because your email is hosted at gmail. There's no way you could guarantee your Message-ID isn't going to be used by gmail.com already. Yes, you could send your default email address via another mechanism, but a quick glance at the code makes me realize that's still used for a bunch of things. We could add another knob, but honestly I'd rather people just use 'random' if the existing logic doesn't work for you. To Mike's other point: >> FWIW, I took a quick look at the MTAs Postfix and Sendmail; Postfix does >> not seem to have any Message-ID-specific configuration knobs, it hardcodes >> adding a Message-ID based on it's idea of the local hostname. Sendmail, >> yes, it looks like you could change it if you really want to; it also >> defaults to something based on the local hostname. I am personally >> skeptical that people actually configure this. > >gethostname() is not the same as what you said we were doing, which takes a >trip through /etc/hosts. Well, technically, it's constructing the Message-ID based on the value of the 'j' Sendmail macro, which is used for a ton of things; that macro value is configurable and in my limited sendmail experience you usually do explicitly configure it (I do not know what that defaults to). --Ken
Re: message-Id has localhost
Ken Hornstein wrote: > Ralph has already noted that this message still has those bytes at the > end, but I think there was a mh-e error as this message wasn't even > clearsigned. Instead this was at the top: >> <#secure method=pgp mode=sign> Yeah, that happens when I futz with the mh-e instructions too many times; they wind up getting ignored. > As a note I'm wondering what those bytes even are; they aren't valid > UTF-8. So the message which I replied to Ralph, I just sent with: send -draftfolder +drafts -draftmessage 9 (it was message 9 in drafts) And it *did not* have the training stuff. I also didn't append my signature, and there may be some other mh-e things that didn't occur. I looked at my .signature.* files, and I don't see the garbage there. It must be something inside emacs, I think. I should replace send with something else and see. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works|IoT architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[ signature.asc Description: PGP signature
Re: message-Id has localhost
Ken Hornstein wrote: >>> 4) Some people, for reasons I would classify as "vague", prefer to >>> generate their Message-IDs locally so their saved copy of the >>> message has the Message-ID in it. >> >> The reason you state seems precise rather than vague. > I mean, that's not a reason in my thinking? Like, WHY do people > want that? That's where things get vague when this came up before. Because, when my message aren't getting through your spam filter, I can refer to the message-Id from my outbox, as a thing you can grep your logs for. (or have your ISP do that) I've also had various ideas about making the message-ID cryptographically strong such that I could recognize when message-ID were really made by me or not. This would help with identifying bounce messages which were really the result of things I sent, vs things where I was impersonated. I think DKIM makes this need obsolete. > FWIW, I took a quick look at the MTAs Postfix and Sendmail; Postfix does > not seem to have any Message-ID-specific configuration knobs, it hardcodes > adding a Message-ID based on it's idea of the local hostname. Sendmail, > yes, it looks like you could change it if you really want to; it also > defaults to something based on the local hostname. I am personally > skeptical that people actually configure this. gethostname() is not the same as what you said we were doing, which takes a trip through /etc/hosts. > My personal feeling is that the people who (a) care about generating a > local Message-ID, and (b) actually care WHAT appears right of the '@' > either need to configure their system appropriately or write code to > change nmh behavior. I'm fine with that. I think that gethostname() is enough. signature.asc Description: PGP signature
Re: message-Id has localhost
Ken Hornstein wrote: > 4) Some people, for reasons I would classify as "vague", prefer to generate > their Message-IDs locally so their saved copy of the message has the > Message-ID in it. I imagine you are looking at a hostly image of me, which sort of fades between higher dimensions :-) > Message-ID locally the default nmh configuration is to try very > hard to get the canonical name for the local system (which isn't > necessarily the same as the user's email domain name, so we can't > use the value of "localname"). Can we just use "localname" from mts.conf?
Re: message-Id has localhost
Hi Ken, > > > 4) Some people, for reasons I would classify as "vague", prefer to > > >generate their Message-IDs locally so their saved copy of the > > >message has the Message-ID in it. > > > > The reason you state seems precise rather than vague. > > I mean, that's not a reason in my thinking? Like, WHY do people want > that? To be able to uniquely refer to that email in future by knowing what the message-id field contains. The reference may be to oneself or to other recipients. That is the purpose of the field. Not knowing the field's value lessens its worth to tracing the flow through downstream parties in log files. It's a lot easier to give the unique message ID than a combination of date, subject, and a bit of a description about the body's content and then have others have to search. -- Cheers, Ralph.
Re: message-Id has localhost
>> 2) The recommendation for Message-IDs is to use a domain name for the >>right-hand side > >Recommendation or rule? I don't recall. Officially, from RFC 5322 §3.6.4: Note: As with addr-spec, a liberal syntax is given for the right- hand side of the "@" in a msg-id. However, later in this section, the use of a domain for the right-hand side of the "@" is RECOMMENDED. Again, the syntax of domain constructs is specified by and used in other protocols (e.g., [RFC1034], [RFC1035], [RFC1123], [RFC5321]). It is therefore incumbent upon implementations to conform to the syntax of addresses for the context in which they are used. [...] The message identifier (msg-id) itself MUST be a globally unique identifier for a message. The generator of the message identifier MUST guarantee that the msg-id is unique. There are several algorithms that can be used to accomplish this. Since the msg-id has a similar syntax to addr-spec (identical except that quoted strings, comments, and folding white space are not allowed), a good method is to put the domain name (or a domain literal IP address) of the host on which the message identifier was created on the right-hand side of the "@" (since domain names and IP addresses are normally unique), and put a combination of the current absolute date and time along with some other currently unique (perhaps sequential) identifier available on the system (for example, a process id number) on the left-hand side. Though other algorithms will work, it is RECOMMENDED that the right-hand side contain some domain identifier (either of the host itself or otherwise) such that the generator of the message identifier can guarantee the uniqueness of the left-hand side within the scope of that domain. >> 4) Some people, for reasons I would classify as "vague", prefer to >>generate their Message-IDs locally so their saved copy of the >>message has the Message-ID in it. > >The reason you state seems precise rather than vague. I mean, that's not a reason in my thinking? Like, WHY do people want that? That's where things get vague when this came up before. >> 7) There's not too much prior art here to crib from because of (3). > >The first-hop MTAs would be a source of prior art. Most probably let >the domain part be given as configuration. Well, I was talking about prior art from MUAs. For MTAs that represent an email domain, it's relatively straightforward because they assume that they're the only one generating Message-IDs for that email domain. FWIW, I took a quick look at the MTAs Postfix and Sendmail; Postfix does not seem to have any Message-ID-specific configuration knobs, it hardcodes adding a Message-ID based on it's idea of the local hostname. Sendmail, yes, it looks like you could change it if you really want to; it also defaults to something based on the local hostname. I am personally skeptical that people actually configure this. And, well, we TRY to use the local hostname to generate the Message-ID for the people who actually want that (because being unique is a MUST in RFC 5322). But as we've seen there's not a fullproof way of doing that. >I think the existing -messageid option which takes either ‘local’ or >‘random’ should also accept ‘@...’ to allow the user to specify it. >This stops using heuristics if the user prefers. My personal feeling is that the people who (a) care about generating a local Message-ID, and (b) actually care WHAT appears right of the '@' either need to configure their system appropriately or write code to change nmh behavior. --Ken
Re: message-Id has localhost
Hi Ken, > 2) The recommendation for Message-IDs is to use a domain name for the >right-hand side Recommendation or rule? I don't recall. > 4) Some people, for reasons I would classify as "vague", prefer to >generate their Message-IDs locally so their saved copy of the >message has the Message-ID in it. The reason you state seems precise rather than vague. > 7) There's not too much prior art here to crib from because of (3). The first-hop MTAs would be a source of prior art. Most probably let the domain part be given as configuration. I think the existing -messageid option which takes either ‘local’ or ‘random’ should also accept ‘@...’ to allow the user to specify it. This stops using heuristics if the user prefers. Instead of having to state that as a default option in the profile for multiple programs, a new profile component, ‘message-id-domain’, could let it be set once. If it is set then -msgid is implied. -- Cheers, Ralph.
Re: message-Id has localhost
Ralph has already noted that this message still has those bytes at the end, but I think there was a mh-e error as this message wasn't even clearsigned. Instead this was at the top: ><#secure method=pgp mode=sign> As a note I'm wondering what those bytes even are; they aren't valid UTF-8. --Ken
Re: message-Id has localhost
>Maybe strcmp(hostname,"localhost") should cause that value to be >ignored, and if necessary, resort to random messageid. Or maybe that should >just be the default in some way. There are a bunch of competing factors here that I am not sure it is possible to resolve to everyone's satisfaction: 1) Message-IDs must be unique 2) The recommendation for Message-IDs is to use a domain name for the right-hand side 3) Most MUAs solve this problem by letting their first-hop MTA generate the Message-ID. 4) Some people, for reasons I would classify as "vague", prefer to generate their Message-IDs locally so their saved copy of the message has the Message-ID in it. 5) Because of (1) and (2), when people configure nmh to generate a Message-ID locally the default nmh configuration is to try very hard to get the canonical name for the local system (which isn't necessarily the same as the user's email domain name, so we can't use the value of "localname"). 6) As seen in this email thread, that doesn't always work, sometimes because of local system configuration for vague reasons. 7) There's not too much prior art here to crib from because of (3). I'm open to suggestions, but I'm not a fan of special-casing localhost. --Ken
Re: message-Id has localhost
Hi Michael, > So it's me sending those bytes, not something in between adding them. ... > I'm clearsigning this email instead. The email I'm replying to: $ hd `mhpath .` | tail -3 1ac0 6e 73 74 65 61 64 2e 0a 0a 0a c0 80 c0 80 c0 80 |nstead..| 1ad0 c0 80 c0 80 c0 80 c0 80 c0 80 0a 0a || 1adc $ So perhaps look at your draft-template files in your `mhparam path`. -- Cheers, Ralph.
Re: message-Id has localhost
On Sat, Dec 30, 2023, 10:50 Michael Richardson wrote: Maybe strcmp(hostname,"localhost") should cause that value to be > ignored, and if necessary, resort to random messageid. Or maybe that > should > just be the default in some way. > I don't think that's a good idea. If the user always wants a random messageid, they can add the switch to their profile. David >
Re: message-Id has localhost
<#secure method=pgp mode=sign> Ralph Corderoy wrote: > Of your three emails in this thread so far, two of them are MIME with a > application/pgp-signature part. Those two have some dodgy bytes after > the PGP signature. This caused some subscribers to the list to not see > your email because their MTAs bounced it. Only two. Weird. I guess I failed to sign one of them? (It's a manual keystroke for me) 1ae0 3d 2d 3d 2d 3d 2d 2d 0a c0 80 c0 80 c0 80 c0 80 |=-=-=--.| 1af0 c0 80 c0 80 c0 80 c0 80 0a 0a|..| So a line of c0 80, and then two line feeds. Looking at my outbox, I also see that. kA0Ds4QFHPC/gYUk/2r4dHM/4Qco5HTdB3rxvTDchkAM1BxEUq7Fog== =b82+ -END PGP SIGNATURE- --=-=-=-- ^@^@^@^@^@^@^@^@ So it's me sending those bytes, not something in between adding them. I'll investigate further. I'm using mh-e and emacs-28.2 The upgrade from emacs-27 to -28 was not without incident... some problems with bbdb byte code compiles eventually fixed with an emacslisp recompile. I'm clearsigning this email instead. ÀÀÀÀÀÀÀÀ
Re: message-Id has localhost
Ken Hornstein wrote: >> 127.0.0.1 localhost obiwan.sandelman.ca obiwan >> >> which is often required for certain things work, but I don't remember >> what now. I'll change it see what breaks. Is there any override? >> Let's see this message. > I see that worked. There is not an override; if you use -messageid random > you'll get a random set of characters as the hostname. I'm personally > reluctant to put yet another knob in nmh just for this. I agree that no more knobs are needed. Maybe strcmp(hostname,"localhost") should cause that value to be ignored, and if necessary, resort to random messageid. Or maybe that should just be the default in some way. signature.asc Description: PGP signature
Re: message-Id has localhost
Hi Michael, Of your three emails in this thread so far, two of them are MIME with a application/pgp-signature part. Those two have some dodgy bytes after the PGP signature. This caused some subscribers to the list to not see your email because their MTAs bounced it. Interesting lines from the first of the two are: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain I noticed that my emails have stuff like: ... --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -BEGIN PGP SIGNATURE- iQFFBAEBCgAvFiEEbsyLEzg/qUTA43uogItw+93Q3WUFAmWPXzARHG1jckBzYW5k ... =b82+ -END PGP SIGNATURE- --=-=-=-- After that last boundary marker comes the weirdness. Here's the end of a hex dump. 1ab0 42 78 45 55 71 37 46 6f 67 3d 3d 0a 3d 62 38 32 |BxEUq7Fog==.=b82| 1ac0 2b 0a 2d 2d 2d 2d 2d 45 4e 44 20 50 47 50 20 53 |+.-END PGP S| 1ad0 49 47 4e 41 54 55 52 45 2d 2d 2d 2d 2d 0a 2d 2d |IGNATURE-.--| 1ae0 3d 2d 3d 2d 3d 2d 2d 0a c0 80 c0 80 c0 80 c0 80 |=-=-=--.| 1af0 c0 80 c0 80 c0 80 c0 80 0a 0a|..| 1afa -- Cheers, Ralph.
Re: message-Id has localhost
>127.0.0.1 localhost obiwan.sandelman.ca obiwan > >which is often required for certain things work, but I don't remember >what now. I'll change it see what breaks. Is there any override? >Let's see this message. I see that worked. There is not an override; if you use -messageid random you'll get a random set of characters as the hostname. I'm personally reluctant to put yet another knob in nmh just for this. --Ken
Re: message-Id has localhost
Michael Richardson wrote: > Let's see this message. Message-ID: <23087.1703897...@obiwan.sandelman.ca> yup. signature.asc Description: PGP signature
Re: message-Id has localhost
Ken Hornstein wrote: > That would be a nmh-style Message-ID; the first number is the process > id of whatever generated that (probably post(8)), the second is a Unix > timestamp. Do you have -msgid set in your profile for send(1)? Yes. But, how to I get it to put in the hostname? I want nmh-style message-IDs so that my outbox will have them. >> I think that mts.conf ought to set this, but localname/localdomain do not >> seem right, and they aren't set, so my hostname (which is correct, I think) >> ought to be used. >> >> I'm running 1.7+dev ... NOPE. >> I'm actually rather unsure how to get a --version out of NMH. > I think pretty much anything takes a -version flag? hmm. -version/-v. not --version. not -V. Silly me. >> Still have @localhost in the message-id. > From the code, that's calling LocalName(1). > What that does is: getaddrinfo(gethostname(), AI_CANONNAME). So maybe > your resolver library canonicalizes your hostname to "localhost"? If > getaddrinfo() fails then it falls back to the value of gethostname(). yes. that's it. 127.0.0.1 localhost obiwan.sandelman.ca obiwan which is often required for certain things work, but I don't remember what now. I'll change it see what breaks. Is there any override? Let's see this message. -- ] Never tell me the odds! | ipv6 mesh networks [ ] Michael Richardson, Sandelman Software Works|IoT architect [ ] m...@sandelman.ca http://www.sandelman.ca/| ruby on rails[
Re: message-Id has localhost
>I noticed that my emails have stuff like: >Message-ID: <25170.1703892488@localhost> > >rather than my hostname or domain name. >Since it's like this in my outgoing folder, it must be generated by NMH >rather than by my local postfix. That would be a nmh-style Message-ID; the first number is the process id of whatever generated that (probably post(8)), the second is a Unix timestamp. Do you have -msgid set in your profile for send(1)? >I think that mts.conf ought to set this, but localname/localdomain do not >seem right, and they aren't set, so my hostname (which is correct, I think) >ought to be used. > >I'm running 1.7+dev ... NOPE. >I'm actually rather unsure how to get a --version out of NMH. I think pretty much anything takes a -version flag? % mhparam -version mhparam -- nmh-1.8 built 2023-06-21 04:53:27 + on Ventura >Still have @localhost in the message-id. >From the code, that's calling LocalName(1). What that does is: getaddrinfo(gethostname(), AI_CANONNAME). So maybe your resolver library canonicalizes your hostname to "localhost"? If getaddrinfo() fails then it falls back to the value of gethostname(). --Ken
