Re: Nmh on MacOs
Hi Eric, > I have mu, nmh, dape, and slime on my emacs I think Emacs users like the MH-E interface to nmh: https://mh-e.sourceforge.io. > Every time I press inc, I get this message This is quite common. https://lists.nongnu.org/archive/html/nmh-workers/2023-03/msg3.html was one ‘recent’ short discussion. There is a text file which tracks your ‘current’ version and when nmh realises it's more new then it says ‘Welcome’. To locate that file, replicate my commands but with the output you get. $ mhparam path context path: mail context: context $ $ cat ~/mail/context Current-Folder: inbox Version: nmh-1.7+dev $ > I know one mistake I made was running inc while in root, but I don’t > know how to fix it. Can you write to that context file? Perhaps nmh is having trouble storing the new version after it has said welcome. I'd expect ~/mail downwards to be owned by you, not root. Or whatever your ‘path’ is shown to be with the above mhparam(1) command. $ cd $ find `mhparam path` ! -user $LOGNAME $ You'll see in the above thread that Ken gives a way of disabling the welcome message but I wouldn't rush to do that in case your problem is a symptom of a mess left by running inc as root. You'd just be disguising the underlying cause and hit a new problem instead. -- Cheers, Ralph.
Re: inc and non-compliant long lines redux
Hi, Ken wrote: > Andy wrote: > > Precisely. Though in my current patch proposal, the problem isn't > > with CR/LF but with the two .. signal terminator coming from POP. > > The two dots must be getting split and the first dot is at the end > > of a buffer and the second is at the beginning of the next buffer. > > Yeah, I was realizing this. I think the logic has to be something > like: > > - netsec_read() into buffer 'b'; > > - Loop over 'b': > > If ! CR && ! LF > putc() > > If CR > check to see if next character is LF. If it is, skip over CR. > If it is not, output CR. If at the end of the buffer, read > more and restart loop. > > If LF > If at the end of buffer, read more and loop back to top. > > Output LF. If the next character is ".", make sure not at end of > buffer (must have at least two more characters). If are, read more > and loop back to top. > > If the next character is "." and the characters AFTER "." is not > CR-LF, skip over ".". If the next characters ARE ". CR LF", then > terminate multiline response. > > I ... think that's correct? This kind of thing gets messy once there starts to be multiple booleans tracking state. Have all cases been considered? I find it easier to have states and events with a table which shows what happens in every S*E cases. So roughly starting like this. Roughly as I don't know the protocol so don't know my aim. But I do know I can shuffle my way through the table, one input byte at a time, finding out the action(s) and possible new state. ══╤═══ │ Event State│ CR LF . else ──┼─── start│ →cr put(LF) put(.) put(*) cr │ put(CR) put(LF),→crlfput(.),→start put(*),→start crlf │ →cr put(LF),→start →crlfdot put(*),→start crlfdot │ put(.),→cr put(.LF),→start →crlf2dot put(.*) crlf2dot │ ... ──┴─── An ‘->err’ can cause an ‘all engines stop’. This doesn't need to be implemented as a big switch statement with an enum for state. If the range of actions is small, it's often possible to use an array of strings, one per state, with the printable chars being a small instruction code. This remains readable, with some study, and represents the table instead of the table ending up unmaintained in a comment and the switch statement spread over screenfuls. -- Cheers, Ralph.
Re: Symbolic link to mhmail
Hello Thomas, kre has given a good answer and I'd do what he suggested, but... > This issue comes from the fact that I wanted to use the > unattended-upgrades package on Debian which uses mailx to send > reports. Since only nmh is detected as providing a compatible mailx > program, mailx redirects to /usr/bin/mh/mhmail. Are you saying that Debian puts in the symlink from /usr/bin/mhmail to /usr/bin/mh/mhmail when selecting nmh as one of the alternative packages to provide a virtual package which unattended-upgrades depends upon? That sounds flawed for the reason you gave. Does Debian have a central place to manipulate PATH to append /usr/bin/mh for unattended-upgrades? apt-file(1) here suggests there's a /lib/systemd/system/unattended-upgrades.service so perhaps a local file to override its PATH? Which version of Debian is this? ‘lsb_release -a’ shows this. -- Cheers, Ralph.
Re: Trivia: space before '(others)' from 'folder' command
Hi Andy,
> Is this perhaps an off-by-one error?
I don't think so.
Try
folders +OpenSourceML | awk '{l=length($1)}l>m{m=l;p=$0}END{print p}'
That it's 81 wide when ‘tput cols’ is 80 is coincidence.
--
Cheers, Ralph.
Re: Trivia: space before '(others)' from 'folder' command
Hi Ken, > > Possibly this could be conditional on the 'all' flag which gets > > turned on if the command name ends in 's'? > > Yeah, that is exactly why it happens. Honestly it doesn't personally > bother me, but what do others think? It seems a bit of a nit which could be groomed out. It was probably just coded like that without considering the multiple uses of the code, as suggested earlier. -- Cheers, Ralph.
Re: Where is my editor?
Hi Andy, > I suppose I could set EDITOR but I've never had to on OpenBSD. POSIX says EDITOR used by more(1) and crontab(1) defaults to vi if not set. But for mailx(1), the default is unspecified unless it's an XSI system when it's ed. mailx's default for VISUAL is vi, XSI or not. That's based on the section-1p man pages here. -- Cheers, Ralph.
Re: Setting send(1)'s draftfolder breaks mhmail.
Hi David, > mhmail currently uses a tmp file, always, for the draft. I think the > fix should depend on if mhmail has been instructed to use send and > send has a draftfolder switch in the profile. In that case it should > create the draft as a new draft message in the draftfolder, instead of > as a tmp file. Or perhaps use -nodraftfolder to cancel any profile setting when send is being used and carry on as before. Though I don't know if -nodraftfolder completely undoes a profile's -draftfolder. -- Cheers, Ralph.
Setting send(1)'s draftfolder breaks mhmail.
Hi, A while ago, I pointed out send(1) ignores the profile's ‘draft-folder’. The suggestion was to duplicate the setting by adding ‘send: -draftfolder drafts’. I think this breaks mhmail(1). mhmail \ -from $from \ -to $to \ -headerfield 'dcc: ralph' \ -subject $subject \ -body '' \ -attach `mhpath $msg` The prescence of -draftfolder causes send: bad message list /home/ralph/mail/mhmailJxHE1P because it changes the interpretation of the argument from a file. This is despite send(1)'s synopsis saying ‘[file...]’. -- Cheers, Ralph.
Re: mysterious c0 80
Hi David, > > > > nmh shouldn't comp(1) a new email today with a NUL in the body, > > > > but it should be able to read and show(1) one. > > > > > > I'm thinking of removing the support in post(8) for sending NULs. > > > Any disagreement? It's not a lot of code so could be easily > > > restored in the future if conditions change. > > > > > > > Now, how about dist(1) of that old email? I'd have thought it > > > > should send the old email verbatim, NUL and all. If that causes > > > > a bounce then the sender can MIME-forward instead with a single > > > > message/rfc822 part. > > > > > > Agreed. > > > > But doesn't dist → send → post so if you remove post's support for > > sending NULs then dist won't be able to send the old email verbatim. > > Yes, but isn't that required by RFC 5322? Yes, RFC 5322 says do not send NULs. > I don't object to violating it in this case, so I'm fine with whatever > we can agree on. I was confused by your ‘Agreed’ above which agreed dist should be able to send and post NULs. -- Cheers, Ralph.
Re: mysterious c0 80
Hi Ken, > It would be nice to understand the root cause of the bug, though. Yes. > So while I agree post would fail with this hypothetical dist(1) of a > message containing a NUL, it's not clear you could inc(1) such a > message in the first place. Today or historically? Historically is a long time. -- Cheers, Ralph.
Re: mysterious c0 80
Hi David, > > nmh shouldn't comp(1) a new email today with a NUL in the body, but > > it should be able to read and show(1) one. > > I'm thinking of removing the support in post(8) for sending NULs. Any > disagreement? It's not a lot of code so could be easily restored in > the future if conditions change. > > > Now, how about dist(1) of that old email? I'd have thought it > > should send the old email verbatim, NUL and all. If that causes a > > bounce then the sender can MIME-forward instead with a single > > message/rfc822 part. > > Agreed. But doesn't dist → send → post so if you remove post's support for sending NULs then dist won't be able to send the old email verbatim. -- Cheers, Ralph.
Re: mysterious c0 80
Hi Ken, > According to RFC 5322, a NUL in a message body is not permitted. > From §3.5: > >body= (*(*998text CRLF) *998text) / obs-body > >text= %d1-9 /; Characters excluding CR >%d11 / ; and LF >%d12 / >%d14-127 obs-body is what allows the NUL. obs-body= *((*LF *CR *((%d0 / text) *LF *CR)) / CRLF) §4: Though these syntactic forms MUST NOT be generated according to the grammar in section 3, they MUST be accepted and parsed by a conformant receiver. nmh shouldn't comp(1) a new email today with a NUL in the body, but it should be able to read and show(1) one. Now, how about dist(1) of that old email? I'd have thought it should send the old email verbatim, NUL and all. If that causes a bounce then the sender can MIME-forward instead with a single message/rfc822 part. -- Cheers, Ralph.
Re: mysterious c0 80
Hi Michael, > hexdump of this draft folder before I hit send: > > 520 2074 6f66 2072 6874 7461 0a2e 430a 756f > 530 646c 6220 2e65 0a0a 000a > > Note 00 0a. If one wrongly encodes NUL as two-byte UTF-8, intended for U+0080 to U+7fff, one gets c0 80. This is the bastard https://en.wikipedia.org/wiki/UTF-8#Modified_UTF-8 -- Cheers, Ralph.
Re: mysterious c0 80
Hi, Ken wrote: > ...draft file does NOT contain a '\n' as the last character? My > memory is that for some strange reason Emacs like to default to doing > that. I suspect we do not test for that. A POSIX text file is zero or more lines where a line is zero or more bytes terminated by '\n'. We shouldn't pander to alien ‘text’ editors and their non-text files. MH-E would presumably ensure or warn if the configuration isn't suitable for Unix? -- Cheers, Ralph.
Re: message-Id has localhost
Hi Andy, > I also find it hard to beleive that someone wants the MUA to have a > specific Message-ID for their email People do all kinds of odd things. Like fully justify text on a media with coarse adjustment. :-) Wanting to specify the whole message-ID field is distinct from wanting to set the trailing ‘@...’ part. - The whole field could just be given as yet another field in the header, though currently nmh will complain it's an ‘illegal header line’. - Setting just the domain part is a more common need. It's tied in to adjusting the From field, and which party takes on the email for delivery. Otherwise, emails from foo.com leak the bar.com identity. -- Cheers, Ralph.
Re: message-Id has localhost
Hi Ken, > I do not believe there is solution to this that will universally work, > or even work in a large majority of cases considering the > configuration of the modern Internet. Agreed. That's why I suggested allowing ‘@...’ in addition to ‘local’ and ‘random’. It allows the user to route around the imperfect heuristics. -- Cheers, Ralph.
Re: mysterious c0 80
Hi Michael, > If it were in my template files, then it would show up in my emacs > buffer and draft files, right? If they're not hiding it from you to be helpful. Try LC_ALL=C sed -n $'/[^\t -~]/l' foo bar ... to see if there is any output for files foo, bar, ... -- Cheers, Ralph.
Re: message-Id has localhost
Hi Ken, > > > 4) Some people, for reasons I would classify as "vague", prefer to > > >generate their Message-IDs locally so their saved copy of the > > >message has the Message-ID in it. > > > > The reason you state seems precise rather than vague. > > I mean, that's not a reason in my thinking? Like, WHY do people want > that? To be able to uniquely refer to that email in future by knowing what the message-id field contains. The reference may be to oneself or to other recipients. That is the purpose of the field. Not knowing the field's value lessens its worth to tracing the flow through downstream parties in log files. It's a lot easier to give the unique message ID than a combination of date, subject, and a bit of a description about the body's content and then have others have to search. -- Cheers, Ralph.
Re: message-Id has localhost
Hi Ken, > 2) The recommendation for Message-IDs is to use a domain name for the >right-hand side Recommendation or rule? I don't recall. > 4) Some people, for reasons I would classify as "vague", prefer to >generate their Message-IDs locally so their saved copy of the >message has the Message-ID in it. The reason you state seems precise rather than vague. > 7) There's not too much prior art here to crib from because of (3). The first-hop MTAs would be a source of prior art. Most probably let the domain part be given as configuration. I think the existing -messageid option which takes either ‘local’ or ‘random’ should also accept ‘@...’ to allow the user to specify it. This stops using heuristics if the user prefers. Instead of having to state that as a default option in the profile for multiple programs, a new profile component, ‘message-id-domain’, could let it be set once. If it is set then -msgid is implied. -- Cheers, Ralph.
Re: message-Id has localhost
Hi Michael, > So it's me sending those bytes, not something in between adding them. ... > I'm clearsigning this email instead. The email I'm replying to: $ hd `mhpath .` | tail -3 1ac0 6e 73 74 65 61 64 2e 0a 0a 0a c0 80 c0 80 c0 80 |nstead..| 1ad0 c0 80 c0 80 c0 80 c0 80 c0 80 0a 0a || 1adc $ So perhaps look at your draft-template files in your `mhparam path`. -- Cheers, Ralph.
Re: message-Id has localhost
Hi Michael, Of your three emails in this thread so far, two of them are MIME with a application/pgp-signature part. Those two have some dodgy bytes after the PGP signature. This caused some subscribers to the list to not see your email because their MTAs bounced it. Interesting lines from the first of the two are: MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" --=-=-= Content-Type: text/plain I noticed that my emails have stuff like: ... --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -BEGIN PGP SIGNATURE- iQFFBAEBCgAvFiEEbsyLEzg/qUTA43uogItw+93Q3WUFAmWPXzARHG1jckBzYW5k ... =b82+ -END PGP SIGNATURE- --=-=-=-- After that last boundary marker comes the weirdness. Here's the end of a hex dump. 1ab0 42 78 45 55 71 37 46 6f 67 3d 3d 0a 3d 62 38 32 |BxEUq7Fog==.=b82| 1ac0 2b 0a 2d 2d 2d 2d 2d 45 4e 44 20 50 47 50 20 53 |+.-END PGP S| 1ad0 49 47 4e 41 54 55 52 45 2d 2d 2d 2d 2d 0a 2d 2d |IGNATURE-.--| 1ae0 3d 2d 3d 2d 3d 2d 2d 0a c0 80 c0 80 c0 80 c0 80 |=-=-=--.| 1af0 c0 80 c0 80 c0 80 c0 80 0a 0a|..| 1afa -- Cheers, Ralph.
Re: nmh is vital to me
Hi dan, > inc produces this infernal message. > the -silent switch doesn't do anything. > how to i shut it off? > i just moved over to debian bookworm See ‘Welcome’ in mh_profile(5) for starters. Though something must be going wrong for it to be shown more than once. Others here will help more, but I thought an initial point in the right direction may help. -- Cheers, Ralph.
Re: Forwarding email
Hi Arthur, > I have been advised, and have tried using: > > forw -mime 5 > > where 5 is the number of the email I am trying to forward. > > This gets me: > > From: aalin...@riseup.net > To: > cqc: > Fcc: +outbox > Subject: Welcome to your The New York Times Replica Edition (fwd) > > #forw [forwarded message] +/home/slaurel/Mail/nytimes 5 > > No draft to edit. What you've shown us above is the draft. If you wish, you may alter what's after the ‘’ to be the body text of your email as normal. Just leave the ‘#forw...’ line at the end. > Sending this gets me an email with that one line: > > #forw [forwarded message] +home/slaurel/Mail/nytimes 5 > > and nothing else. That's because the draft isn't being processed by mhbuild(1). I assume you're seeing the whatnow(1) prompt and are using that to send the email? Before you send it, enter ‘mime’ to have whatnow run mhbuild. This will alter the draft. (You can see the change by entering ‘edit’ at the same prompt to return to the editor. Exit the editor without making changes.) Send the email in the same way as before. -- Cheers, Ralph.
Re: Forwarding html email
Hi Arthur, > It doesn't work when forwarding to a Proton mail account. Does Proton reject the whole email? Or does it appear in the inbox at Proton? > I guess i will have to contact the people at Proton for help. There's also https://www.reddit.com/r/ProtonMail/ The key information is your MIME email contains a part with MIME type message/rfc822. mhlist will show this too if you forward to yourself. > I want you to know it is always appreciated. Thank you. -- Cheers, Ralph.
Re: Forwding html email
Hi Arthur, > I believe i asked this once before when I was having difficulty > forwarding html email. The answer I received was to type "mime" at the > "what now" prompt and then send. There are gaps in that explanation of what you do. Rather than pick over the fine detail of your set-up, I'll initially suggest trying something simple. You're not trying to forward HTML email. You're trying to forward an email, any email, as an email rather than a textual representation of it. Attaching the email to a new MIME email as a message/rfc822 part does this. An easy way to do this is ‘forw -mime 42’ where 42 is the email's number. You will see a ‘#forw ...’ at the end of the draft. After editing the draft, at the ‘What now?’ prompt type ‘mime’ before sending it in the normal way. The recipient should be able to treat the attached email as an email in its own right. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Conrad, > Do the packaging systems on the affected operating systems support > "recommends" or "suggests"? My impression is the only OS where this is a problem is macOS. And if it has a packaging system then groff isn't in it. So instead of FreeBSD's ‘install package groff’ prompt when mandoc has trouble, Apple plough on and use mandoc. The one escape Apple do offer is their man program checks for groff and uses that in preference to mandoc. So nmh can either recommend groff be obtained on macOS or if nmh is packaged in something like Homebrew then that's the point where a ‘depends’ or ‘recommends’ could be used. -- Cheers, Ralph.
Re: Number of nmh users.
Hi, M. Levinson emailed me off list but has agreed for the content to be forwarded because I thought it was of general interest. > >nmh-workers has ~170. > > And for what it's worth, one of those subscriptions is an automated > relay to the gmane.mail.nmh.devel newsgroup, so the total number of > readers may be larger than the number of direct subscribers to the > mailing list. > > Admittedly the number of people who are still using NNTP newsreaders > in 2023 may also not be very large. But there could be some overlap > between gmane's user base and nmh's user base if they both appeal to > the sort of people who prefer old-fashioned software. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Anthony, > Not all groff parts were commented out. It will use groff if > installed and if mandoc gives an unsupported warning. > > https://github.com/apple-oss-distributions/man/blob/man-46/man/man.sh Thanks, that's interesting. Line 375 onwards, https://github.com/apple-oss-distributions/man/blob/man-46/man/man.sh#L375 shows how they #ifdef'd __APPLE__ to stop the simple ‘install groff’ message. That message matches Oliver's description of FreeBSD. Instead, they echo a warning to stderr and then sleep for a second to let it be seen. How annoying that delay must be each time one opens the same man page. And from a company that prides themselves on UI. :-) -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Bakul, > > groff was the macOS package Ken mentioned that provided troff. > > I've no knowledge of how Apple ship different troffs and how one can > > be specified as an nmh dependency. > > Apple doesn’t ship any *roff any more. If you look at their > /usr/bin/man script, they commented out anything to do with groff and > only rely on mandoc. Odd. That doesn't tally with what Ken said earlier. https://lists.nongnu.org/archive/html/nmh-workers/2023-04/msg8.html - Tell MacOS X users to install groff(1) (the man(1) script will try to call groff if it encounters a warning like this from mandoc) Perhaps he's referring to a different man script provided by another implementation or a different version of macOS? -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Anthony, > > And if not now, then it may do in the future, > > Pure FUD. In your anger, you cut too much. > > It may be formatting other things wrongly. > > And if not now, then it may do in the future, > > especially if knowedgable man-page authors continue to write nmh's > > *man pages*, not mandoc-input. I was not referring to mandoc having a regression, as I think I make clear. A future change to nmh's man pages might use something unsupported by mandoc. Just as .fc has been unsupported all this time. Unless, that is, the edict is past that nmh's man pages are now mandoc pages, or even mdoc pages. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Paul, > What happened to the possibilitiy of simply replacing the .fc macros, > and the use of tbl, with tab characters? Or one of the other alternatives I gave along with that one. But it still leaves the risk of mandoc not coping with something else now or in the future. macOS, for that's its current marketing capitalisation, scripts the possibility of a troff being installed to override mandoc formatting. Seems mandoc's failure to be troff and its common preprocessors is an accepted problem by Apple. Use the solution they provide. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi kre, > > - Require groff. A win all round. > > If you mean "require a *roff compat text processor" then OK, but if > you really mean "groff" then no, never. groff was the macOS package Ken mentioned that provided troff. I've no knowledge of how Apple ship different troffs and how one can be specified as an nmh dependency. I also didn't notice anything in nmh's Git which looks like a package specification explicitly for macOS. > MH (and nmh) is a nice free, available to everyone, mail system. > Removing that by requiring anything encumbered by any kind of > commercial licence (including the GPL) would be a huge step into the > darkness. I only require nmh has a troff to format its man pages. Anything else is by definition inferior and not a man-page formatter. By the time wannabees become a man-page formatter, they have become troff and all its common preprocessors. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Ken, > > The status quo is fine. It doesn't require understanding all of > > troff. Just man(7) plus the odd bit here and there. > > Sigh. The "odd bit" unfortunately, for me, requires a lot of > knowledge that seems to take some serious roff-fu. Which is why I said refer to the mailing list for help. Which you cut. > Let's take the example you gave where the first line for a man page > that uses tbl should contain: > > '\" t > > So, my question is ... what does this mean? It's a magic string recognised by man-page processors, like man(1). They hand off the man page to troff for formatting but need to know what preprocessors to run in advance. These are programs which expand some of the man page into troff language. tbl is one. pic(1) is another common one for diagrams. The agreement is the man page author specifies this by convention: the first line being the three characters quote, backslash, double-quote and a list of characters. man(7) here lists: e eqn(1) g grap(1) p pic(1) r refer(1) t tbl(1) v vgrind(1) > I understand that \" is a comment, but I'm confused about the leading > single quote. Ignore all troff documentation until you have seen Ossanna's and Kernighan's CSTR 54, the ‘Troff User's Manual’. https://troff.org/54.pdf Just skim the first few pages and you'll see it starts with lots of handy tables referring to the following pages. The paper is the high quality succinctness one expects from Bell Labs and Kernighan. It's a delight to read. I still refer to it often. The very first paragraph of the main text answers your question. 1. General Explanation 1.1. Form of input. Input consists of text lines, which are destined to be printed, interspersed with control lines, which set parameters or otherwise control subsequent processing. Control lines begin with a control character — normally . (period) or ’ (single quote) — followed by a one or two character name that specifies a basic request or the substitution of a user-defined macro in place of the control line. The control character ’ suppresses the break function — the forced output of a partially filled line — caused by certain requests. The control character may be separated from the request/macro name by white space (spaces and/or tabs) for aesthetic reasons. Names should be followed by either space or newline. Control lines with unrecognized names are ignored. So «'» and ‘.’ are equivalent in many cases. The ‘\"’ comment above should be on a request line rather than a text line as a text line is ‘destined to be printed’, even if empty. $ printf '%s\n' '\" t' foo '.pl \n(plu' \" t foo .pl \n(plu $ $ printf '%s\n' '\" t' foo '.pl \n(plu' | nroff | cat -n 1 2 foo $ printf '%s\n' '.\" t' foo '.pl \n(plu' | nroff | cat -n 1 foo $ printf '%s\n' \''\" t' foo '.pl \n(plu' | nroff | cat -n 1 foo $ > As a random note, this string is rather hard to search for. CSTR 54 mentions it at the start of the second page of documentation. Request Summary B Request normally causes a break. The use of ’ as control character (instead of .) suppresses the break function. And then 1.1, above, covers it again. What's ‘filling’? Well, the index of chapters and the requests each covers has 4. Text Filling, Adjusting, and Centering > there's a huge pile of knowledge that is assumed you know. Given you don't know what you don't know, I can say you're wrong. It's not a huge pile. > And I do it so infrequently that I have to re-learn it all every time > I want to edit a man page. Just copy what's already there. > (I did play around with tbl, and it seems like that is actually very > easy so I am thinking that Anthony is is right and we should just > switch to that). You're just prolonging the agony. Make nmh on macOS depend on macOS's troff. Whether that's a virtual package satisfied by others or an actual package if there's only one. You've identified one thing mandoc is formatting wrongly. It may be formatting other things wrongly. And if not now, then it may do in the future, especially if knowedgable man-page authors continue to write nmh's *man pages*, not mandoc-input. -- Cheers, Ralph.
Number of nmh users. (Was: Unsupported nroff macros on MacOS X)
Hi doug, > There are 26 members on Savannah. Is that all of us that are left? The number of subscribers to the mailing lists may be a better representation. nmh-announce has ~60. It goes down whenever there's a flurry of announcements due to domains having expired, accounts being deleted, or, quite often, a misconfigured SMTP server. nmh-workers has ~170. Debian's statistics suggest 60 use package nmh regularly out of those taking part in the poll. https://qa.debian.org/popcon.php?package=nmh -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Ken, > I see what you mean; you can't mix and match macros across packages. ...across macro sets like man and mdoc. No. The status quo is fine. It doesn't require understanding all of troff. Just man(7) plus the odd bit here and there. It's sufficed for nmh for ages. Any occasional issue can be discussed on the list. man(7) still serves well for many other mainstream packages with copious man pages. Silencing the .fc warning has easy fixes. - Require groff. A win all round. - Replace .fc with plain tabs if that gives suitable results for all .fc uses. - Experiment with the .HP, .IP, or .TP man macros to see if they're suitable. - Introduce tbl(1). $ tbl | nroff .TS tab(,); l l. foo,bar a longer foo,and another bar .TE .pl \n(plu foobar a longer foo and another bar ^D $ -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Ken, I don't have time to reply fully now. > My specific question is: should we replace the .fc macros in nmh man > pages with mdoc macros? The question is flawed. You could replace .fc plus *all* the man macros with all-mdoc macros in all the nmh man pages. It's man ^ mdoc == 1. But that would be a daft waste of time. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Ken, > Are we all in agreement that (when possible) we should change man > pages over to mdoc and new man pages should be written in mdoc? No, see my other email which crossed. -- Cheers, Ralph.
Re: Unsupported nroff macros on MacOS X
Hi Ken, > This manpage is not compatible with mandoc(1) and might display > incorrectly. mandoc is a pain. It's one of many programs which attempt to interpret man pages whilst being an incomplete implementation. I hang out in places which like to talk about troff/nroff, including for man pages, and mandoc's flaws crop up a lot. > After some digging, it turns out man(1) is a shell script and to make > a long story short is running this command: > > mandoc -Tlint -Wunsupp I'd suggest disabling that if it's possible. > And some quick googling suggests that in fact the mandoc macros I understand it's a program which tries to interpret files intended for a troff/nroff rather than a set of macros. > .fc aint one of them. Ignore mandoc. :-) > I will admit that my roff-fu is not very good, but I took a look at > this. It seems this is a common idiom for nmh man pages. > Specifically (this is from packf(1) but it's similar everywhere else): > > .SH "PROFILE COMPONENTS" > .fc ^ ~ > .nf > .ta 2.4i > .ta \w'ExtraBigProfileName 'u > ^Path:~^To determine the user's nmh directory > ^Current\-Folder:~^To find the default current folder > ^Msg\-Protect:~^To set mode when creating a new `file' > .fi > > So as I vaguely understand it, the .fc line sets '^ as a field > delimeter and '~' as the character to pad a field. Yes, i.e. ‘~’ is where to share out the stretchable space within the field. > .nf sets the following lines to no-fill mode. Yes, so don't join input lines together to form output lines. > ".ta 2.4i" sets the tab stop to 2.4 inches. Yes, just a single tab stop. $ printf '%s\n' '.ta 2.4i' $'a\tb\tc\td' '.pl \n(nlu' | nroff a bcd $ > The following line sets the tab stop to the width of > "ExtraBigProfileName" and 'u is the default horizontal span for the > terminal Not quite. The single tab stop is set to the width of the string ‘ExtraBigProfileName ’ including a couple of spaces. \w'' gives the width of the string in ‘basic units’. The ‘ta’ request assumes numbers are in ems so the ‘u’ overrides this to show the number is in basic units. > (now that I look at it, I'm not sure why there are two .ta lines right > after another). Agreed. > This isn't wonderful and I'd like to fix it, but I'm not sure what > to do. Ideas include: > > - Tell MacOS X users to install groff(1) (the man(1) script will try > to call groff if it encounters a warning like this from mandoc) This will fix things for macOS users viewing non-nmh too. +1 > - Switch to some other roff construct; I was under the impression that > actual tabs would work? I'm not sure why tabs aren't used here. Perhaps a carry over from more complex layouts in other pages which use the same valid technique. > - Switch to tbl(1) macros which as far as I can tell are supported by > mandoc and seem to work everywhere. tbl(1) is a preprocessor which reads the man-page source, leaves most well alone, and turns what's in between lines of .TS to .TE into roff for processing by troff/nroff. Other programs which read man pages do try to emulate it as part of processing the file, typically with the same gotchas and noise as they trigger with the rest of the file. man pages are defined as what troff/nroff handle with the man(7) set of macros. It's a complex language. Pretenders start on a sticky wicket. The .TS and .TE are copied through allowing a macro set, e.g. man, to define them if it wants. This allows further processing of tbl's output which now lies between them. A man page which uses tbl should indicate this by the first line being '\" t so man knows to use tbl(1) as well as troff/nroff. > Given my druthers I think I'd rather do the last one, since this kind > of seems like a table! It's not the least work. You may find you have to then muck around with tbl's commands a bit to get a nice table for ASCII and UTF-8 on the TTY and PostScript/PDF for the page or graphical viewers. Others have mentioned the mdoc macros. These are an alternative to man. Their big selling point is semantic mark-up rather than presentation. But they have their own problems, they're YA-standard, and I'd avoid them. Stick with simple man macros and troff/nroff with ASCII, UTF-8, and PostScript/PDF as targets. -- Cheers, Ralph.
Re: flist -- "Killed" -- oom (*not* 1.8 related)
Hi Ken,
> > > I think we have to push this back on the MH-E people
...
> >$ refile +31415 >$ folder +31415
> >31415+ has 1 message (1-1).
>
> I'm aware of that, but what happens if you have a subfolder that is
> all numeric? I believe all of the nmh tools will treat that subfolder
> as a message
$ ref +3/1/4/1/5
Create folder "/home/ralph/mail/3/1/4/1/5"? yes
$ folder +3/1/4/1/5
3/1/4/1/5+ has 1 message (1-1).
$ scan -forma %{from} +3/1/4/1/5 1
Ken Hornstein
$ scan -forma %{from} +3/1/4/1
scan: unable to read: Is a directory
scan: scan() botch (-3)
$
> (that's the real issue).
The real issue is nmh doesn't forbid folders named with just decimal
digits and even creates them when requested. MH-E is set a bad example.
--
Cheers, Ralph.
Re: flist -- "Killed" -- oom (*not* 1.8 related)
Hi, Ken wrote: > I think we have to push this back on the MH-E people; Robert's > suggestion to add a non-numeric prefix to directories it creats sounds > like the best answer to me. $ refile +31415
Re: flist -- "Killed" -- oom (*not* 1.8 related)
Hi Greg, > $ valgrind flist -sequence xyz -all ... > ==729147== Warning: set address range perms: large range [0x59c8e040, > 0xa313d8ac0) (undefined) To watch progress, try strace -e openat flist -recurse -sequence xyz -all >/dev/null Also, is there anything odd about your tree of directories? Does it have cycles? Symlinks? Is it all on local disk? -- Cheers, Ralph.
Re: way to use Bcc: with mhmail?
Hi Tom, > Is there a way to Bcc: with mhmail? Experiment with one of -headerfield 'bcc: Tom R ' -headerfield 'dcc: Tom R ' > I do not want any destination addresses in the header. You may have seen ‘undisclosed-recipients:;’ in the To field. https://en.wikipedia.org/wiki/Blind_carbon_copy -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, > Then I think that we should do the same in 1.8 for all platforms. Your principles soon went out the window. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, I've had one night's sleep spread over the last two nights so I'm not going to have time for nmh until the weekend. > And my feeling at this point is to put out an RC3 with the 1.7 > behavior for $HOME. Which, to be clear, is not the behaviour Ken and kre want. :-) Stephen, I'd appreciate knowing if unsetting HOME with RC2, e.g. ‘env -u HOME ...’, is sufficient for xlbiff's test harness. $ HOME= MH=$PWD/mh_profile mhparam path mhparam: environment variable HOME is empty $ $ env -u HOME MH=$PWD/mh_profile mhparam path /tmp -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi, I wrote: > If 3 is implemented then the user may have thought a different > pretence would occur. After all, it's an arbitrary choice. Perhaps > he expects it will carry on just like his shell does. This has overlap with DWIM, ‘do what I mean’, and Perl's TMTOWTDI, ‘there's more than one way to do it’. DWIM obviously gets one tied to chair. http://www.jargon.net/jargonfile/d/DWIM.html I'm a fan of Perl, programming a lot of it in Perl 4 and early 5 days, but having introduced it to a large site of engineers who did a bit of programming, I came to see TMTOWTDI as a problem. I could read all their code because I knew all the ways. They wrote in dialects by using different subsets, reducing re-use and understanding. Many saw TMTOWTDI as a problem. It's one reason why Python has boomed. Amongst its ‘import this’ Zen are Explicit is better than implicit. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. An empty HOME isn't explicit in its meaning, it's ambiguous, we shouldn't guess, and the obvious way of specifying the home directory is to set HOME to it. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, > > This close to a release, I think we should stick with requiring HOME > > to be non-empty if it's set as otherwise there's too many paths to > > consider which the test harness probably doesn't exercise. > > I'd rather crank out an RC3 than pass up the opportunity to solidify > the behaviour here. The behaviour cannot solidify further. :-) It's clear, determinate, predictable, and simple to document. I thought RCn+1 was meant to be minimal differences from RCn? That's why Andy Bradford's POP3 patch is post 1.8. Switching the behaviour of a set-but-empty HOME now seems significant given some value for HOME must be chosen. And will mean another round of testing by others on top of your time. > Ken's point that different treatmet of unset vs empty HOME is a mistake > is fairly compelling for me. I've covered that in my reply to kre just now. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi kre, > There is never a need to deliberately make anything an error to > satisfy user expectations, if the user wants a complaint about HOME > being an empty string, rather than some other behaviour, all they need > is one of... My stance has nothing to do with satisfying ‘user expectations’ so I'll ignore this aspect. > > No, one cannot say of an unset HOME that it may be set by accident. > > No, but it may have been unset by accident, where the intended value > for mh to use was something different than the pw_dir entry. No-one > can possibly know in that case, any more than in the case where > HOME='' (I think you've over-cropped in general.) There is a marked difference which I thought I had made very clear. There are three cases. All may be accidental or deliberate, obviously. 1. - HOME is unset. - This is valid, i.e. POSIX allows it. - The only route for continuing is it to use a mandatory passwd.pw_dir. - HOME=passwd.pw_dir results. 2. - HOME is set and non-empty. - Use it. - If that causes errors then die, do not try again with passwd.pw_dir. - HOME=$HOME results. 3. - HOME is set but empty. - There are at least three ways of continuing. - Pretend HOME=$PWD - Pretend HOME=/ - Pretend HOME=passwd.pw_dir - Other alternatives include trying HOME=... until one ‘works’. - HOME=??? results. There is no obvious way to continue for 1 and 2 other than the above. The result is not surprising. Even if the manual isn't read. If 3 is implemented then the user may have thought a different pretence would occur. After all, it's an arbitrary choice. Perhaps he expects it will carry on just like his shell does. This could result in surprising files being read, e.g. /tmp/.mh_profile, and unwanted commands being run, e.g. through a profile's ‘moreproc’ or ‘mhshow-show-text/plain’. (top(1) had a similar issue. It used $PWD when HOME was unset or empty allowing a privilege escalation through its configuration file; cue CVE.) Turning it about. What is the user trying to achieve by explicitly setting an empty HOME? Whatever it is, he can achieve it in an *unambiguous* manner by unsetting HOME or setting it to a non-empty value. It seems you're arguing for him to instead first refer to the manual to determine what set-but-empty means to nmh in the hope it matches his need when he could just take and explicit short-cut? > There is always ambiguity if you are doing something different than > what is documented to happen. That is not ambiguity. That is error. mh_profile(5) can easily say HOME must be non-empty if set for those that read the fine manual. > The question is what is best for nmh - and for that I agree with Ken, > treating an unset HOME and HOME='' differently is a mistake. > All that is required to make that OK is to document it. No, you must decide which of the many alternatives of 3 above is the ‘nmh way’, document that, test it, and have some users be surprised by the result. There is no ‘All that...’. Ken, you, and David all seem irked by unset and set-but-empty being treated differently, as if you'd like a binary outcome by first conflating the ternary input to binary. There isn't a clear method of merging those two inputs. Ternary's messy, as ever. We can't stop HOME being unset or being empty so there are three safe outcomes for three inputs. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi Ken,
> > What's the intent of an empty HOME?
> > Is it set by accident when it's meant to be unset?
> > Is it empty by accident when it's meant to be non-empty?
> > Do they want HOME=/, HOME=$PWD, or are they expecting it to error.
> > Any choice could be not what the user intended so exit.
>
> I mean ... you could say the exact same things of an unset HOME!
No, one cannot say of an unset HOME that it may be set by accident.
Nor that it is empty by accident since it is neither empty or non-empty.
This being Unix, the user knows what they're doing and POSIX seemingly
allows it so we permit the lack of HOME and use pw_dir.
There is no ambiguity over the right course.
As is often the way with an environment variable, HOME overrides pw_dir
just as an argv option overrides an environment variable which overrides
a configuration-file setting.
But if HOME is empty we do not know their intent so to ignore it and use
pw_dir may not be what they think will occur. The wrong profile could
be read or the wrong .netrc used, upsetting the user.
> My point is that I cannot see a reason to treat unset HOME and empty
> HOME differently in nmh; if unset HOME uses pw_dir, then I would argue
> that an empty HOME should do the same thing.
I disagree. In the same way that seeing ${foo:-bar} instead of
${foo:bar} in sh is a smell.
> I was curious and some system call traces suggest that what happens
> under the hood is that when 'cd' is called with an empty HOME sh
> explicitly calls chdir(getcwd()).
Sometimes. I think it's more complex than that, e.g. perhaps PWD has
an effect, and it will obviously vary widely.
$ LC_ALL=C env HOME= PWD=/nonexist strace -fe getcwd,stat,chdir dash -c cd
stat("/nonexist", 0x7ffc101d1bb0) = -1 ENOENT (No such file or
directory)
getcwd("/home/tmp/1628693513.257008819", 4096) = 31
chdir("/home/tmp/1628693513.257008819") = 0
+++ exited with 0 +++
$
$ LC_ALL=C env HOME= PWD=$PWD strace -fe getcwd,stat,chdir dash -c cd
stat("/home/tmp/1628693513.257008819", {st_mode=S_IFDIR|0755, st_size=4096,
...}) = 0
stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
chdir("/home/tmp/1628693513.257008819") = 0
+++ exited with 0 +++
$
$ LC_ALL=C env -u PWD HOME= strace -fe getcwd,stat,chdir dash -c cd
getcwd("/home/tmp/1628693513.257008819", 4096) = 31
chdir("/home/tmp/1628693513.257008819") = 0
+++ exited with 0 +++
$
--
Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi Ken, > > So an unset HOME is allowed by this function, it's an empty HOME > > which isn't. > > It strikes me as strange that there is a difference between an unset > HOME and an empty HOME in terms of behavior. I mean, yes, I can see > how the code is written, the historical precedent and how we got here, > but ... well, I'm trying to understand the justification for treating > those differently. A non-empty HOME is clear; its content should be used. An unset HOME is clear; it cannot be used. There's a ready fallback, getpwuid(3) and pw_dir so use that. What's the intent of an empty HOME? Is it set by accident when it's meant to be unset? Is it empty by accident when it's meant to be non-empty? Do they want HOME=/, HOME=$PWD, or are they expecting it to error. Any choice could be not what the user intended so exit. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi kre,
Thanks for your learned input on this. As I said in another reply just
now, where I listed set_mypath(), HOME being unset is fine as getpwuid()
is the fallback in which case pw_dir must be non-empty.
> > Alexander does point out that HOME is supposed to be valid according
> > to POSIX,
>
> That's actually not correct, all the text quoted requires is that HOME
> be set at login time - nothing at all about what can be done with it
> after that.
Agreed. Nearby, is:
8.1 Environment Variable Definition
...
Some are placed into the environment by the implementation at the
time the user logs in; all can be added or changed by the user or
any ancestor of the current process.
Does ‘changed’ include unsetting them? I'd expect the spec. to be
precise and list ‘unset’ to clarify if they can.
> Further POSIX reading will also find, in the description of ~
> expansion in the shell (XCU 2.6.1 in Issue 7 TC2 - the current version
> from 2018):
>
> If the login name is null (that is, the tilde-prefix contains only
> the tilde), the tilde-prefix is replaced by the value of the
> variable HOME. If HOME is unset, the results are unspecified.
...
> Setting HOME='' isn't even mentioned: in that case ~/foo expands to
> /foo (which is actually what you want
Agreed, it isn't mentioned.
> Similarly, in XCU 4 in the description of the cd utility:
>
> 1. If no directory operand is given and the HOME environment
> variable is empty or undefined, the default behavior is
> implementation-defined and no further steps shall be taken.
>
> Again, HOME being unset is allowed there, this time with
> implementation-defined results (which just means the implementation
> must say what happens in the doc).
Yes, but also allowed there is ‘empty’ which also triggers
implementation-defined.
Also, POSIX's chdir(3) says
int chdir(const char *path);
...
ERRORS
The chdir() function shall fail if:
...
ENOENT A component of path does not name an existing
directory or path is an empty string.
chdir(NULL) is invalid, obviously. But so is chdir("") so if an empty
HOME is really to be treated as the root directory then code can't do
h = getenv("HOME");
r = chdir(h ? h : "/");
but must instead
r = chdir(h && *h ? h : "/");
I expect lots of non-nmh code doesn't do that extra check so the chdir()
has no effect and the error return may not even be tested so an empty
HOME results in the current directory being used rather than /.
> Running with HOME unset is not even all that unusual
I think it is a bit. :-)
> "env -i command" results in that
Agreed, my self-at(1)-ing shell scripts run at(1) under ‘env -i’ but
bequeath HOME, LANG, LOGNAME, and PATH at the same time.
> (and I'd even retain: relative to / if HOME is not set, which it can
> be when there as been no login - like when you're in single user
> mode).
It would be nice for nmh to work in single-user mode as reading email
might be vital in fixing problems. As long as HOME is unset and pw_dir
is valid then all is well at the moment.
> I'd also suggest never issuing errors because the user's environment
> isn't like we think it ought to be, unless that actually prevents
> something functioning as intended.
The code I replaced mislead the user as to the cause of the exit.
That's fixed and no HOME is okay. The possible errors are now
environment variable HOME is empty
password entry not found
[perhaps with ‘getpwuid() failed’ and errno if it did]
password entry has empty home directory
This close to a release, I think we should stick with requiring HOME to
be non-empty if it's set as otherwise there's too many paths to consider
which the test harness probably doesn't exercise.
> But I agree that a relative Path in MH should be relative to HOME
Yes, I think it's too late to alter that.
Though ‘path: mail’ would be a more Unixy default. :-)
--
Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi az,
> "Author: Ralph Corderoy
> Date: Thu May 13 13:46:20 2021 +0100
>
> sbr/path.c: add set_mypath() to factor out repeated code."
I think it's worth expanding on that.
commit d8ca46fabc26469be325b73a73dcc26e70681eb5
Author: Ralph Corderoy
Date: Thu May 13 13:46:20 2021 +0100
sbr/path.c: add set_mypath() to factor out repeated code.
set_mypath() sets the existing global ‘mypath’ which really holds
the home-directory's path. As getenv(3) and getpwuid(3)'s result
can be invalidated by further library calls, a value they return is
mh_xstrdup()'d; not all the old bits of code replaced by a call to
set_mypath() were doing this.
Also add more explicit error messages if set_mypath() has trouble
finding the home directory's path.
And here's the current code I wrote for set_mypath() which is causing
the trouble.
/* set_mypath sets the global mypath to the HOME environment variable if
* it is set and not empty, or else the password entry's home-directory
* field if it's found and not empty. Otherwise, the program exits
* with an error. The value found is copied with mh_xstrdup() as later
* library calls may invalidate returned values. */
void
set_mypath(void)
{
char *var = getenv("HOME");
if (var) {
if (!*var)
die("environment variable HOME is empty");
mypath = mh_xstrdup(var);
return;
}
errno = 0;
struct passwd *pw = getpwuid(getuid());
if (!pw) {
if (errno)
adios("", "getpwuid() failed"); /* "" prints errno! */
die("password entry not found");
}
if (!*pw->pw_dir)
die("password entry has empty home directory");
mypath = mh_xstrdup(pw->pw_dir);
}
So an unset HOME is allowed by this function, it's an empty HOME which
isn't.
An unset HOME falls back on getpwuid() which must dish up a non-empty
pw_dir. I think that's reasonble.
> a quick glance seems to indicate that only context_read in
> sbr/context_read.c looks at $MH for finding a home
And uip/install-mh.c in a similar manner.
> but it does (well attempts) that just a few lines AFTER set_mypath has
> bailed out on it...
>
> i think we need to fix that sequence up a little :-)
It's the long-standing arrangement going back to Git's
docs/historical/mh-6.8.5/sbr/m_getdefs.c which allows an empty HOME but
not an empty pw_dir, though it complains that the later is the lack of
a HOME ‘envariable’. :-)
if (mypath == NULL) {
if (mypath = getenv ("HOME"))
mypath = getcpy (mypath);
else
if ((pw = getpwuid (getuid ())) == NULL
|| pw -> pw_dir == NULL
|| *pw -> pw_dir == 0)
adios (NULLCP, "no HOME envariable");
else
mypath = getcpy (pw -> pw_dir);
if ((cp = mypath + strlen (mypath) - 1) > mypath && *cp == '/')
*cp = 0;
}
It looks to me like code assumes mypath isn't NULL, e.g. exmaildir(),
so not bothering to call set_mypath() if MH is set doesn't look a goer.
At least not without a considerably more analysis which would probably
inspire changes which would ‘disappoint’ RC-maker David. :-)
--
Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, > I'd like to get Ralph's take on what we should do. Thanks. If my suggestion to Stephen of unsetting HOME works and is acceptable to him then I suggest we don't change nmh for this release. > > A further documentation issue: mh-profile(5) does not specify the > > treatment of a relative Path. I expected it to be relative to the > > directory of the profile file, but it seems to actually be relative > > to HOME. > > Right, it's relative to HOME. I'll add this clarification to the man > page: > A relative Path is relative to the user's $HOME directory. > unless there's another suggestion. As HOME needn't be set how about mh_profile(5) saying Path: Mail Locates nmh transactions in directory “Mail”. +A relative path starts from the user's home directory. This is the only mandatory profile entry. (profile, no default) I see the ENVIRONMENT section doesn't mention HOME. Perhaps, HOME If set, it is used as the user's home directory. If not set then getpwuid(3)'s pw_dir is used. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi Stephen, > I have investigated the failure of the xlbiff tests with nmh 1.8RC2. Thanks. > $ printf 'Path: /tmp\n' > /tmp/mh-profile-minimal > $ HOME= MH=/tmp/mh-profile-minimal /usr/bin/mh/mhparam path My suggestion for a quick fix to try is to not have HOME in the environment so getpwuid(3) is used. $ HOME= MH=$PWD/mh_profile mhparam path mhparam: environment variable HOME is empty $ $ env -u HOME MH=$PWD/mh_profile mhparam path /tmp > This is a regression. HOME is used only to set the default profile > file to "$HOME/.mh_profile". But nmh doesn't need HOME if MH is set. I think nmh *as a whole* does need HOME to be non-empty if it's set. -- Cheers, Ralph.
Re: 1.8RC2?
Hi David, > Alexander wrote: > > https://qa.debian.org/excuses.php?package=nmh > > It looks like that dependent package is xbiff. Do you know how that > is identified as a dependecy? I don't see it as an explicit > dependency in nmh itself. Finding a Debian system, I think it's more that package xlbiff depends on nmh. $ dep=Depends,PreDepends,Suggests,Recommends $ apt-rdepends -r -f $dep -s $dep nmh Reading package lists... Done Building dependency tree... Done Reading state information... Done nmh Reverse Depends: exmh (1:2.9.0-2) Reverse Depends: xlbiff (4.5.2-1) exmh Reverse Suggests: nmh (1.7.1-7) Reverse Recommends: muttprint (0.73-10) muttprint xlbiff $ Using apt-cache(1) to look forwards, I see: muttprint Enhances: mail-reader, news-reader Recommends: mutt | sylpheed | gnus | xfmail | exmh | mail-reader xlbiff Depends: nmh exmh Depends: nmh | mh, mime-support nmh Conflicts: mh Provides: mail-reader, mh Recommends: default-mta | mail-transport-agent Replaces: mh Suggests: exmh -- Cheers, Ralph.
Re: nmh latest on Mageia 8
Hi Jerry, > + Creating dir /home/jerry/code/nmhgit/nmh/test/testdir/tmp/ Have you got something odd with mkdir(1) which would make it print that message, especially with a ‘-p’? I don't think ‘Creating dir’ appears in nmh's source. -- Cheers, Ralph.
Re: [nmh-commits] [SCM] The nmh Mail Handling System branch, master, updated. 1.8-RC1-9-g68228e3c
Hi Ken, > man/burst.man: re-word to avoid ‘digestifying’, etc. ... > when the email is sent out about the change I get them a little > mangled because the script that notifies about changes doesn't mark > that email as UTF-8. Agreed, it doesn't. They arrive as valid UTF-8 here which show just fine so I hadn't noticed a problem, but it's clearly wrong. I expect it's a bug in the script but have forgotten where is it to be found. > It looks like you CAN mark the encoding of commits using the config > variable i18n.commitEncoding, although this suggests the default is > UTF-8 and when I look at all the commit encodings they are blank, so > maybe I don't quite understand it? git-commit(1) covers it. The default for the log message is UTF-8. If the i18n.commitEncoding you gave is set then its value is copied into the commit's header. If that header isn't present then the log message is UTF-8. > I don't know if that would change the encoding marked in the > notification email. I doubt it. I think the script probably pre-dates the popularity of UTF-8. :-) -- Cheers, Ralph.
Re: [SCM] The nmh Mail Handling System branch, master, updated. 1.7-branchpoint-887-g54434563
Hi David, > > I can see tag 1.8-branchpoint is the branch point for the > > 1.8-release branch, but why are 69027ab9, 760a6ba9, 5608cda8, which > > alter VERSION and DATE, on master rather than 1.8-release? > > Good question. When I did that I was thinking of where master is at > that point in time. Maybe it should just be 1.8+dev? Yes, I think so if all further 1.8 work happens on branch 1.8-release. I think that's what docs/README.developers suggests too, even before my recent commit. :-) -- Cheers, Ralph.
Re: nmh 1.8?
Hi Andy, > > Has anyone had a chance to review my proposed changes to inc to be > > able to handle long lines from POP sources? Is the latest in Git? I see the ‘andy-long-line-patch’ at http://git.savannah.nongnu.org/cgit/nmh.git/refs/ with one commit: http://git.savannah.nongnu.org/cgit/nmh.git/commit/?h=andy-long-line-patch=08dffe5aba9563ff06e82d2a8e6a6d746223d10f -- Cheers, Ralph.
Re: [SCM] The nmh Mail Handling System branch, master, updated. 1.7-branchpoint-887-g54434563
Hi David, > > Should I stop pushing commits now so as to not get in your way, or > > keep pottering them in as they may make it if there's an 1.8-RC2? > > I'd rather avoid code changes because, in theory, they'd require > complete re-testing. Agreed. > If you find a code change that really should go in, how about posting > it here for concurrence? Everything else should be easy to cherry > pick. I've pushed a few more trivial documentation things to master, which don't have to make 1.8. I'm not intending to post anything to branch 1.8-release, considering it your domain. I'm a bit puzzled by the tags and branches though. ┌ 68228e3c (origin/master, origin/HEAD) │ man/burst.man: re-word to avoid ‘digestifying’, etc. ⋮ ├ a3a23a66 man/*.man: fix options which didn't escape their hyphen. ├ 277347a4 NEWS: fix spelling of ‘MIME’. ├ f8fb242b NEWS: add a link to Norm's Wikipedia page. ├ 19d63f85 .gitignore: move ‘/configure~’ entry to correct section. ├ 5608cda8 Update for 1.8-RC1. │ ├ 760a6ba9 (tag: 1.8-RC1) │ Updates for 1.8 release. ├ 69027ab9 Update for 1.8 release. │ │ ┌ a60e1455 (origin/1.8-release) │ │ man/*.man: fix options which didn't escape their hyphen. │ ├ 8ec6dc0e NEWS: fix spelling of ‘MIME’. │ ├ 1f4822e8 NEWS: add a link to Norm's Wikipedia page. │ ├ 2e344864 .gitignore: move ‘/configure~’ entry to correct section. ├─┘ ├ 54434563 (tag: 1.8-branchpoint) Updates for 1.8 release. ├ bee25da0 Ignore configure tilde file ⋮ I can see tag 1.8-branchpoint is the branch point for the 1.8-release branch, but why are 69027ab9, 760a6ba9, 5608cda8, which alter VERSION and DATE, on master rather than 1.8-release? -- Cheers, Ralph.
Re: nmh 1.8?
Hi Kevin, > Is anyone here packaging nmh as part of a .deb file in the process of > testing? Alexander Zangerl is Debian's packager in the past; I'm CC-ing him. The end of docs/README.developers says Keep an eye on Debian's packaging, especially what patches they have to apply, and the results of their Lintian checker, which even includes spelling errors in man pages and binaries. https://sources.debian.net/src/nmh/1.6-16/debian/patches/ https://lintian.debian.org/full/a...@debian.org.html#nmh Perhaps some nmh developer that uses Debian, or Ubuntu?, could provide package-building commands, including lintian(1), for Makefile.am so Lintian's complaints are known before release. I assume if we grabbed the debian directory, e.g. https://sources.debian.org/src/nmh/1.7.1-12/debian/, then we could have a stab at building the .deb on a Debian machine; I have one to hand. I've had a look at the Lintian output and fixed a spelling mistake. It also says debian/control could do with a Homepage definition: Alexander, it's https://www.nongnu.org/nmh/ That README also mentions a quick overview of what's shipping what version: A useful overview of what third parties are shipping which release is available at https://repology.org/project/nmh/versions with a quick overview on the Badges tab which shows https://repology.org/badge/vertical-allrepos/nmh.svg. -- Cheers, Ralph.
Re: send(1) and Draft-Folder.
Hi David, > > Ralph wrote: > > > > Draft-Folder: To specify the default draftfolder > > > > > > > > Removing the hyphen from the second draft-folder helps associate > > > > it with the -draftfolder switch, I think. > > > > > > Is this equivalent? > > > > > > Draft-Folder: -draftfolder's default > > > > I don't like that, because then I have to figure out how -draftfolder > > fits in. > > I might have come across as harsh on that. I didn't find that. :-) I was just left a bit unclear on your point but I think it's that I might understand draft folders, from mh-draft(5), but not know about this command's -draftfolder, say, and so it adds another level of indirection to get to the answer. > In reality, I don't have a preference. Given its peers, e.g. whom(1)'s Path:To determine the user's nmh directory Draft-Folder:To specify the default draftfolder Aliasfile: For a default alias file postproc:Program to post the message it may as well be left alone. It looks like there could be a general sweep through them to make them more brief and simply refer to an option where applicable. So for those above, Path:The user's nmh directory Draft-Folder:-draftfolder's default Aliasfile: -alias's default postproc:A post(8)-like program which does the work This would make more sense if the man pages listed options rather than burying them in prose. -- Cheers, Ralph.
Re: [SCM] The nmh Mail Handling System branch, master, updated. 1.7-branchpoint-887-g54434563
Hi David, > Summary of changes: > NEWS | 112 > - I've pushed a little update to that to master which falls after the 1.8 tags. a3a23a66 (origin/master, origin/HEAD) man/*.man: fix options which didn't escape their hyphen. → 277347a4 NEWS: fix spelling of ‘MIME’. f8fb242b NEWS: add a link to Norm's Wikipedia page. 19d63f85 .gitignore: move ‘/configure~’ entry to correct section. 5608cda8 Update for 1.8-RC1. 760a6ba9 (tag: 1.8-RC1) Updates for 1.8 release. 69027ab9 Update for 1.8 release. 54434563 (tag: 1.8-branchpoint, origin/1.8-release) Updates for 1.8 release. bee25da0 Ignore configure tilde file 2f95bdd7 Explicitly mark mhical.c as living in $(srcdir) f116eb31 uip/mhlistsbr.c: switch some printf(3) for puts(3), and similar. Should I stop pushing commits now so as to not get in your way, or keep pottering them in as they may make it if there's an 1.8-RC2? (It's been so long, I've forgotten the conventions.) -- Cheers, Ralph.
Re: nmh 1.8?
Hi David, > HEAD builds and tests cleanly for me on Fedora, Solaris 11, and > Cygwin. Good to hear. I had a ‘All 118 tests passed’ with 1.7-branchpoint-884-gf116eb31 when doing ‘NMH_VALGRIND=1 VALGRIND_ME=1 make check’ on a friend's quicker PC running Manjaro. > Do you or anyone else have anything else you'd like to put in before > starting the 1.8 release cycle? Nothing which should hold it up. I'm dabbling with mh-chart.man and the bash-completion generation at the moment, fixing a little bit of degradation, but it way well not meet the cut and that wouldn't bother me. -- Cheers, Ralph.
Re: nmh 1.8?
Hi Ken, > Sigh. Ralph, you and I don't agree on Content-MD5, which is fine. The issue is not the repeated distractions of only I used the options or what was the point of them anyway. It's that functionality was deleted but at the same time documented as deprecated and the options left in as no-ops so users wouldn't cotton on. The better path given your opinion would have been to take a stand, delete the code and the options and say it's just tough on the users. That's effectively where we are now as I've removed the options and lingering documentation. -- Cheers, Ralph.
Re: send(1) and Draft-Folder.
Hi David, > How about changing all of these man page descriptions: > > Draft-Folder: To find the default draft-folder > > to: > > Draft-Folder: To specify the default draftfolder > > Removing the hyphen from the second draft-folder helps associate it > with the -draftfolder switch, I think. Is this equivalent? Draft-Folder: -draftfolder's default -- Cheers, Ralph.
Re: valgrind Complaint for test/mhshow/test-charset.
Hi David,
> I don't get that complaint here, Fedora 37 with glibc 2.36.
Good to know. The problem appears on a friend's Manjaro system, an Arch
Linux spin-off, also with glibc 2.36.
> > ==19813551992192== Invalid read of size 8
> > ==19813551992192==at 0x4024AC0: strncmp (strcmp-sse2.S:160)
> > ==19813551992192==by 0x4004EBF: is_dst (dl-load.c:216)
> > ...
> > ==19813551992192==by 0x4923707: iconv_open (iconv_open.c:39)
>
> I would think that would be suppressed by this in test/valgrind.supp:
>
> {
>iconv_open on Fedora 33
>Memcheck:Addr16
>fun:strncmp
>...
>fun:iconv_open
> }
That's for an invalid address when reading sixteen bytes whereas it says
‘size 8’ above. I'll add something similar for Addr8.
> And it's clean for me on a non-debug build with all of the compiler
> checks that build_nmh enables.
I've got a few warnings that I've been committing fixes for.
--
Cheers, Ralph.
valgrind Complaint for test/mhshow/test-charset.
Hi, I'm getting a repeatable valgrind(1) complaint with NMH_VALGRIND=1 make TESTS=test/mhshow/test-charset check on one machine but not the other. Both are x86_64 but with different C libraries, etc. I think it's start_test 'Encoded parameter value' with the interesting, repetitive, bit of valgrind: ==19813551992192== Invalid read of size 8 ==19813551992192==at 0x4024AC0: strncmp (strcmp-sse2.S:160) ==19813551992192==by 0x4004EBF: is_dst (dl-load.c:216) ... ==19813551992192==by 0x4923707: iconv_open (iconv_open.c:39) ==19813551992192==by 0x116A93: get_param_value (mhparse.c:3957) ==19813551992192==by 0x116D54: get_param (mhparse.c:3906) ==19813551992192==by 0x116D54: content_charset (mhparse.c:3433) ==19813551992192==by 0x1184B7: convert_content_charset (mhshowsbr.c:1264) ==19813551992192==by 0x1184B7: show_content_aux (mhshowsbr.c:373) ==19813551992192==by 0x1186B7: show_text (mhshowsbr.c:551) ==19813551992192==by 0x118E19: show_single_message (mhshowsbr.c:178) ==19813551992192==by 0x119135: show_all_messages (mhshowsbr.c:141) ==19813551992192==by 0x10F147: main (mhshow.c:404) ==19813551992192== Address 0x4b80161 is 1 bytes inside a block of size 8 alloc'd ==19813551992192==at 0x4841888: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) ==19813551992192==by 0x40248AE: malloc (rtld-malloc.h:56) ==19813551992192==by 0x40248AE: strdup (strdup.c:42) ==19813551992192==by 0x40063D4: decompose_rpath (dl-load.c:629) ... ==19813551992192==by 0x4923707: iconv_open (iconv_open.c:39) ==19813551992192==by 0x116A93: get_param_value (mhparse.c:3957) ==19813551992192==by 0x116D54: get_param (mhparse.c:3906) ==19813551992192==by 0x116D54: content_charset (mhparse.c:3433) ==19813551992192==by 0x1184B7: convert_content_charset (mhshowsbr.c:1264) ==19813551992192==by 0x1184B7: show_content_aux (mhshowsbr.c:373) ==19813551992192==by 0x1186B7: show_text (mhshowsbr.c:551) ==19813551992192==by 0x118E19: show_single_message (mhshowsbr.c:178) ==19813551992192==by 0x119135: show_all_messages (mhshowsbr.c:141) ==19813551992192==by 0x10F147: main (mhshow.c:404) It looks to me like the problem lies outside of nmh but I'm wary since that way lies ‘It's a compiler bug!’ so I thought I'd share with the list. -- Cheers, Ralph.
Re: nmh 1.8?
Hi Paul, > > - The generation and verification of Content-MD5 headers is no > > longer performed. The -check and -nocheck switches to various nmh > > programs that would control this functionality still exist, but > > are non-functional and will be removed in the next release. > > > > That it not deprecation. The user who thinks the MD5 is being > > checked is being fooled by them silently becoming a no-op. They > > should be removed so the user instead suffers an error and learns > > of the change. I'll try to get this done. > > Removing the options entirely doesn't seem like deprecation either. No, it's not. But it's better than the introduced security flaw. > Seems like they should maybe emit warnings for a release Yes, i.e. be deprecated. But that ship has sailed and I don't think Ken would be pleased if I added them back in so they could be deprecated. -- Cheers, Ralph.
Re: nmh 1.8?
Hi David, > Greetings as we approach the new year. Yes, hello all. Hope you've had a nice Christmas. > What does everyone think of pushing out a 1.8 soon? +1 > Here are changes since 1.7.1: > https://git.savannah.nongnu.org/cgit/nmh.git/plain/docs/pending-release-notes --- DEPRECATED FEATURES --- - The generation and verification of Content-MD5 headers is no longer performed. The -check and -nocheck switches to various nmh programs that would control this functionality still exist, but are non-functional and will be removed in the next release. That it not deprecation. The user who thinks the MD5 is being checked is being fooled by them silently becoming a no-op. They should be removed so the user instead suffers an error and learns of the change. I'll try to get this done. -- Cheers, Ralph.
Re: send(1) and Draft-Folder.
Hi David, > > I've Draft-Folder set to drafts in .mh_profile. Using at(1) to > > ‘send 42’ fails because `mhpath +`/42 doesn't exist. > > Would it succeed with 'send -draftfolder drafts 42'? Yes, I've just tried ‘send -draftfolder drafts 42’ and it worked. > Or if your profile included '-draftfolder drafts' for a 'send' > component? I was about to add that originally when I wondered about Draft-Folder and noted the man page said it was used. > > The man page seems inconsistent. The default is -nodraftfolder so > > why is Draft-Folder listed under Profile Components as it isn't > > used? > > Yeah, this looks like an unfortuate result of tacking on the draft > folder facility, It would have been easier to have had that from the > beginning. And only used that, because the old-style simple draft > file isn't very useful with it. So as it stands, the bug is with the man page and Draft-Folder should be removed. > > I'd expect a .mh_profile Draft-Folder to be used by default. > > I agree. But I'm not sure if we want to change it at this point, to > not break existing users' workflows. This is a perennial problem. I normally bring up Python's ‘from future import ...’ method of requesting new behaviour. https://lists.nongnu.org/archive/html/nmh-workers/2014-08/msg00129.html ‘How about the opposite of Python's `from future import ...'? Have a ~/.mh_profile entry like `old-fogey' or `stuck-in-1989' as a global decision maker when code changes to support both old and new ways.’ https://lists.nongnu.org/archive/html/nmh-workers/2014-08/msg00089.html ‘Like Python's `from future import sanity'. ‘A problem with ~/.mh_profile calling for sanity is that it might be wanted generally but the user can't be sure no scripts they use want insanity?’ All ideas welcome, however daft. > For new users, install-mh could add a "Draft-Folder" component and add > a corresponding -draftfolder to a "send" component. That's an improvement on now. :-) -- Cheers, Ralph.
send(1) and Draft-Folder.
Hi, send [-help] [-version] [-alias aliasfile] [-draft] [-draftfolder +folder] [-draftmessage msg] [-nodraftfolder] ... The -draftfolder +folder and -draftmessage msg switches invoke the nmh draft folder facility. This is an advanced (and highly useful) feature. Consult mh-draft(5) for more information. PROFILE COMPONENTS ... Draft-Folder:To find the default draft-folder ... DEFAULTS ... `-nodraftfolder' I've Draft-Folder set to drafts in .mh_profile. Using at(1) to ‘send 42’ fails because `mhpath +`/42 doesn't exist. ‘send drafts/42’ does work. The man page seems inconsistent. The default is -nodraftfolder so why is Draft-Folder listed under Profile Components as it isn't used? I'd expect a .mh_profile Draft-Folder to be used by default. Correction welcome if I'm not understanding this. I don't use send(1) directly often, mainly just as ‘s’ from whatnow(1). -- Cheers, Ralph.
Re: nmh setup on macos by newbie
Hi Torsten, Ken's already answer some of your questions... > so this is my first reply to a mailinglist post - I addressed it to > you (reply to from:), cc'ed the list and hope for the best. Yep, that'll work fine. > > I think it is the problem. Try adding -initialtls to inc's options, > > either by editing ~/.mh_profile or just temporarily at the command > > line where it will augment those in the mh_profile(5). > > > > inc -initialtls > > I tried that and now inc is knocking at the door but: > > inc: TLS negotiation failed: error:1408F10B:SSL > routines:ssl3_get_record:wrong version number Perhaps you're trying to establish a TLS connection on a plain-text port. As Ken said, the server often offers a TLS version on TCP port 995 in addition to the plain-text one on 110. $ egrep -w 'pop3s?' /etc/services pop3 110/tcp pop3 110/udp pop3s 995/tcp pop3s 995/udp $ > Should I insist on getting inc to fetch from the remote host? Or is > the fetchmail setup good enough? I use fetchmail(1) because when I set all this up, nmh's inc(1) couldn't talk POP3. > ... it seems that inc does not see new mail that procmail 'delivered' > into inbox - only when I us C-u M-x mh-rmail and choose the inbox > folder with 'all' do I get to see all mails, including the unread > ones. Ken's explain inc has nothing to do; procmail has done inc's job for it. There are MH-E users on the list so hopefully they can chip in with what keystrokes will ‘re-scan’ the folder so you see what procmail's delivered, if you stick with having it deliver to the nmh folder. > ... they seem to be the ghosts of the list emails that have been > sorted by procmail into their mailinglistfolders? but they show in ls > and have weight as in take space? Are they in both places - respective > listfolder and inbox? I wouldn't have thought they're leftovers from procmail. See rmm(1) for details of rmmproc and refile(1) for -link to use a hard link. ‘ls -li ,42’ will show the link count as the third word and tell you if the comma file is a hard-link to another copy of the email elsewhere. > I am reading this list across the years and the manual (rand-mh book) > plus the MH-E info file in emacs. Jerry Peek's book is very good, it's what I learnt from. The man pages also attempt to detail every option and many options have been added since Peek's book was written. -- Cheers, Ralph.
Re: nmh setup on macos by newbie
Hi Torsten, > I hope I can ask for guidance here. Yes, very much so. > inc: -ERR [SYS/PERM] This account requires an encrypted network > connection to login. Your current connection is vulnerable to > villainous voyeurs. Reconnect using transport layer security, aka > SSL/TLS, to access this account. ... > inc: -host example.com -user u...@example.com ... > machine example.com > login user > password Password > > How do I tell inc to use encryption/tls/ssl, if that is the problem... I think it is the problem. Try adding -initialtls to inc's options, either by editing ~/.mh_profile or just temporarily at the command line where it will augment those in the mh_profile(5). inc -initialtls There are two ways of establishing a TLS connection to a server: - Immediately after the TCP connection is established. - After both the TCP connection has been established and some unencrypted protocol has been exchanged to request the upgrade to encryption using TLS. I've suggested the first method in the hope your server supports that. If it doesn't seem to, at least on the port you're trying, then inc has a -tls option which attempts the second method. Lastly, there's a -snoop option which may help diagnose how far things proceed. > I hope I did everything right. Perfectly. -- Cheers, Ralph.
ali(1)'s glob Matching is Too Slack.
Hi, I was just pootering about adding comments at the start of functions in uip/aliasbr.c to document what they do, which means working out what they do, and noticed this. $ ali -alias <(echo 'foo*bar: xyzzy') foonot xyzzy $ I think the ordinary punter would expect ‘foo*bar’ to need ‘bar’ to match at the end. mh-alias(5) says In match, a trailing “*” on an alias will match just about anything appropriate. ... news.*: news ... Lastly, “news.anything” is aliased to “news”; the full stop is just another literal character. My ‘*’ in the example above isn't trailing but still globs. My comment so far is: /* Does string match aliasent where the latter may contain a ‘*’ which * matches zero or more characters and stops matching, * e.g. "foobar" matches "foo*xyzzy". */ static bool aleq (char *string, char *aliasent) I think the fix is to allow only a single ‘*’ which must be the last character but this may break some user's habits. -- Cheers, Ralph.
Re: inc and non-compliant long lines redux
Hi, Ken wrote: > Andy wrote: > > > Is this only seen with inc(1) reading from a particular source? > > > > I believe the only problem that I found was inc and reading from a > > POP source; sorry for not being more clear. Specifically in > > popsbr.c and pop_getline(), it would call netsec_readline() which > > uses a buffer of 65535 for holding data, and then pop_getline() > > would stuff it into a buffer that was BUFSIZ (1024 on my system), > > thus truncating whatever was returned to 1024. > > Officially such a message violates RFC 5322 (line length limit of 998 > plus CR LF), which means it violates the POP3 RFC as well. I think the POP3 line-length limit is 512 including CR LF. That's what fetchmail(1) enforces, and that has been pointed at a lot of servers over the year. For the message sent back in answer to RETR, fetchmail uses 8192. A quick look at some RFCs gave: Post Office Protocol - Version 3 https://www.rfc-editor.org/rfc/inline-errata/rfc1939.html 3. Basic Operation ... Responses in the POP3 consist of a status indicator and a keyword possibly followed by additional information. All responses are terminated by a CRLF pair. Responses may be up to 512 characters long, including the terminating CRLF. POP3 Extension Mechanism https://www.rfc-editor.org/rfc/rfc2449.txt 4. Parameter and Response Lengths ... The maximum length of the first line of a command response (including the initial greeting) is unchanged at 512 octets (including the terminating CRLF). > But I suspect we just have to deal. Sigh. It's one peer of one user. No one else suffers. (Sorry, Andy.) Seems a shame to complicate the code. > netsec_readline() can handle a maximum line length of 32k, which I > chose based on it being way larger than what email protocols say is > the limit of a "line". My only question is: should there be a > hardcoded limit in terms of line length, or should we have the limit > be whatever you can malloc()? As I said back when this issue first arose, I think unbounded other than by what's available is a terrible idea as it's handing over control of whether the machine starts thrashing to a third party. So that leaves either: - Telling Andy he's on his own. - Complicating the logic of receiving lines using Andy's patch. - Upping the truncation limit to avoid truncating Andy's case. Andy, if it were the last, what kind of size would be needed, i.e. how long is the longest line from this brain-damaged peer, including CR LF? -- Cheers, Ralph.
Re: inc and non-compliant long lines redux
Hi Andy, > Along the way I found 2 other bugs. Not only does inc fail with long > lines that are greater than 32768 bytes, it also truncates lines that > are less than that value and longer than 1024 bytes (not sure if this > behavior is intentional, but it seems like an oversight to me). Is this only seen with inc(1) reading from a particular source? >From a text file, ‘inc -file foo’, it seems fine. -- Cheers, Ralph.
Re: Misplaced header include in some files for sbr/globals.h?
Hi Andy, > #include "sbr/globals.h" > > For some reason it's nested in an #ifdef block, and I think it should > actually be outside of the block. Yes, it went wrong in a commit which added #include'd that new sbr/globals.h to many files. Two of the files had problems. I seem to have corrupted the commit message; it should have the [...] bit added: commit 58cbb3bc57ef21309e7e852e122e8dbd46530e12 Author: Ralph Corderoy Date: Sat Nov 12 10:58:21 2022 + rcvtty.c, slocal.c: remove condition on #include-ing globals.h. sbr/globals.h was only #include'd if HAVE_GETUTXENT was defined. The #include was added in the wrong place due to using the last existing [#include as a guide. The problem] wasn't noticed until Andy Bradford spotted it. Fixes 472d87ae. -- Cheers, Ralph.
Re: inc and non-compliant long lines redux
Hi Andy, > Not sure where they went... they used to be defined in mh.h: And elsewhere. They went with C99 being a requirement. commit c08cb60fe34b42b0a83bc88b57fd41b07793863f Author: Ralph Corderoy Date: Mon Sep 4 10:07:35 2017 +0100 Replace FALSE and TRUE with C99's false and true. Remove the duplicate definitions of FALSE and TRUE. Change a few variables from int to bool at the same time. The semantics of bool are different to int because `bool b = 42' maps 42 to 1, but if the code was relying on that then it needs shaking out anyway. -- Cheers, Ralph.
Re: sortm error analysis
Hi Dave,
> Running sortm on a folder with about 6000 messages, sortm reports "can't
> parse date field" on two messages. I've added -verbose but have no clue
> to what is wrong with the date field on those messages.
...
> Date: Wed, 21 Sep 2022 06:47:55 -0400
> Date: Wed, 21 Sep 2022 03:49:19 -0700
If I create a couple of emails here and replace their Date fields with
those values then they scan and sortm fine, so that's odd.
$ scan -forma '%(tws{date}) %(pretty{date})' 1 2
Wed, 21 Sep 2022 06:47:55 -0400 Wed, 21 Sep 2022 06:47:55 -0400
Wed, 21 Sep 2022 03:49:19 -0700 Wed, 21 Sep 2022 03:49:19 -0700
$
$ sortm -verbose 1 2
sorting by datefield date
$
$ mv 1 3
$ sortm -verbose 2 3
sorting by datefield date
renaming message chain from 3 to 2
message 2 becomes message 3
$
Does the above scan command work with your emails?
> There are no hidden characters or trailing whitespace.
To be sure, can you give us the output of this command, changing the
message numbers 2 and 3 to be your two problem emails.
$ cd `mhpath`
$ grep -ib '^[[:space:]]*date[[:space:]]*:' 2 3 | sed -n l
2:389:Date: Tue, 30 Nov 1999 16:10:20 +$
3:1426:Date: Thu, 15 Mar 2001 18:27:11 -0500$
$
What's the output of ‘sortm -version’.
Perhaps the problem is not with those fields, but some internal
corruption of them by the time they're being processed, which is why
grep has -b above.
--
Cheers, Ralph.
Re: 'all' folder msgs from last 3 weeks; of symlinks?
Hi Tom, > I have many nmh folders, that get filled by procmail. I'd like to have, an > 'all' > folder that is built by a shell script that could be called by cron daily. > I'd be happy, with all msgs, say from the last 21 days. Perhaps this script > would remove bad symlinks in 'all' each time it runs. You're implying you want the ‘files’ in +all to be symlinks rather than hard links? refile(1) can make hard links with -link. I'd have thought useful parts would be: rmf +all folder -create +all folder +foo && pick -list -after -21 | xargs -rd\\n refile -link +all folder +bar && pick -list -after -21 | xargs -rd\\n refile -link +all sortm +all If you want symlinks and you want to preserve +all from one run to the next but remove the symlinks to files which are now too old whilst adding those which are new then comm(1) would be handy. > **In this way I could use sort and 'scan' to list the nmh 'all' folder > msgs sorted by time.** One has wondered about the separation of email files into folders and sequences, the overlap between the two, and most commands only working within a single folder. -- Cheers, Ralph.
Re: Surprising MIME Type from Android.
Hi David, > > Something for mhfixmsg(1) to correct? > > At this point, I think the trade-off favors manually fixing the > apparently rare offending messages rather than investing effort in > enhancing mhfixmsg. Fair enough. Either way, I think nmh should baulk at invalid characters in the MIME type so to avoid creating ‘42.2.*’ as it contains shell globbing characters. -- Cheers, Ralph.
Re: Surprising MIME Type from Android.
Hi Paul, Haven't noticed a comment in a References field before: References: <20220722115151.80a8221...@orac.inputplus.co.uk> <20220725092852.e055e20...@orac.inputplus.co.uk> (sfid-20220725_052923_626513_424143A1) <20220725102256.65c6c2e0...@grass.foxharp.boston.ma.us> > >pick -sea 'content-type.*\*' > > I have one. Message came from a hotmail user, via outlook.com. > > $ mhlist > msg part type/subtype size description > 2122 multipart/mixed 968K > 1 multipart/alternative 859 > 1.1 text/html 342 > 1.2 text/plain 69 > 2 image/* 715K 20220609_075908_resized.jpg The email which triggered it here is from a user who typically uses Outlook, though all the ‘android’ bits of this particular email pushed me away from that. I also noticed its Message-ID field was created on my machine when it arrived. Thanks to Bob too for checking. I followed my own suggestion with interesting, in a bad way, results. In my +inbox, there are no culprits according to grep(1) as all the hits are false positives. $ LC_ALL=C grep -i 'content-type.*\*' [0-9]* 6105: document-response = Content-Type [ Status ] *other-field NL 81708: > pick -sea 'content-type.*\*' 81714:> > Content-Type: image/*; name=3D"20220721_180552.jpg" 81714:> pick -sea 'content-type.*\*' $ But if I use pick(1) to do the same search: $ pick -sea 'content-type.*\*' 4 hits $ scan lp 6105 13-12-08 meThe Common Gateway Interface (CGI) 8717 20-02-19 Mark Sapiro Re: [Mailman-Announce] I18n for Mai 81708 22-07-25 Paul Fox Re: Surprising MIME Type from Andro 81714+ 22-07-25 Bob Carragher Re: Surprising MIME Type from Andro $ Message 8717 is a hit, but it shouldn't be: $ LC_ALL=C grep -i content-type 8717 'content-type:multipart/signed': 0.07; 'bay': 0.09; 'content- Content-Type: multipart/mixed; boundary="===2295464778554599938==" Content-Type: multipart/signed; micalg=pgp-sha1; Content-Type: multipart/mixed; boundary="ttQpQgr8FL6th88C6FQswkN3KhxwqEQFl" Content-Type: text/plain; charset=utf-8 Content-Type: application/pgp-signature; name="signature.asc" Content-Type: text/plain; charset="us-ascii" $ Either it's some nmh corruption of the input as it's buffering it, or a fault in -search's regexp processing. I'd guess the former. See, this is why it's worth not glossing over these RFC violations! :-) -- Cheers, Ralph.
Re: Surprising MIME Type from Android.
Hi again, > I ran mhstore(1) and was surprised by the ‘81081.2.*’ filename. ... >2 image/* 2985K ... > I haven't checked yet, but I assume it violates the RFCs. It does by my reading of https://datatracker.ietf.org/doc/html/rfc2045#section-5.1 Something for mhfixmsg(1) to correct? > _com.samsung.android.email_135816585189000 > Content-Type: image/*; name="20220721_180552.jpg" ... > which suggests a Samsung Galaxy S9 took the photo but that doesn't meant Has anyone else got an example? pick -sea 'content-type.*\*' Note, don't use ‘--content-type’ as that only searches the header's fields. -- Cheers, Ralph.
Surprising MIME Type from Android.
Hi, I ran mhstore(1) and was surprised by the ‘81081.2.*’ filename. $ mhlist -nov p=1 msg part type/subtype size description 81081 multipart/mixed 4036K 1 multipart/alternative 3143 1.1 text/html 1327 1.2 text/plain 780 2 image/* 2985K $ I haven't checked yet, but I assume it violates the RFCs. (Note also how it's assumes the ‘msg’ column only needs four digits.) The relevant part of the email: _com.samsung.android.email_135816585189000 Content-Type: image/*; name="20220721_180552.jpg" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="20220721_180552.jpg"; size=2985255 The JPEG contains exif:Model: SM-G960F which suggests a Samsung Galaxy S9 took the photo but that doesn't meant it was the Android which sent the email. On a related note, I've been putting lines like mhshow-show-application/vnd.openxmlformats-officedocument.wordprocessingml.document: %lfile '%f' | \> mhshow-show-application/vnd.openxmlformats-officedocument.spreadsheetml.sheet: %lfile '%f' | \> into ~/.mh_profile and was thinking it would be nice to have fall-backs like mhshow-show-application/vnd.openxmlformats-officedocument.*: mhshow-show-application/vnd.*: mhshow-show-application/*: mhshow-show-app*: mhshow-show-*: where the last character can be ‘*’ to indicate a prefix match of the type or subtype. If used for the type then the subtype can't be given. I thought this would be a backwards-compatible extension due to ‘*’ not being valid in a MIME type. :-) -- Cheers, Ralph.
Re: mhl linewrapping
Hi Philipp, > First of all it depends on the terminalsize, if the size is not given > by the arguments. This leads to diffrent linewrapping on reply > depending on the size of the terminal. This could be fixed by going > to some default width when stdout is not a tty. It would seem right to only use the terminal to set the width when the output is a TTY. There is already a default width of 80, mentioned in mhl(1), if the terminal width can't be found so my initial thought it that would do. > Next it only supports hard linewrapping. Therefor sometimes words and > links get split. Some support for softwrapping would be nice. So mhl > could search for the last whitespace befor the selected width. Are you just talking about the body component or all of them? Are you aware of -fmtproc, etc? Some versions of fmt(1), for example, do other nice things like trying to avoid the word ‘I’ at the end of the line. Other formatters might re-introduce two spaces after the end of a sentence. $ cat mhl.format from: to: subject: : body:component="",format,formatarg=-42,formatarg=-c,nowrap $ $ /usr/lib/nmh/mhl -form ./mhl.format -fmtproc fmt -width 42 <`mhpath .` from: Philipp Takacs to: nmh-workers@nongnu.org subject: mhl linewrapping Hi The linewrapping in mhl is not realy good and I would like to implement some improvements. First of all it depends on the terminalsize, if the size is not given by the arguments. This leads to diffrent linewrapping on reply depending on the size of the terminal. This could be fixed by going to some default width when stdout is not a tty. Next it only supports hard linewrapping. Therefor sometimes words and links get split. Some support for softwrapping would be nice. So mhl could search for the last whitespace befor the selected width. What do you think about this? Philipp $ -- Cheers, Ralph.
Re: Formatting annotations in mhl
Hi kre, > Does anyone have a good (or even passable) way to write an mhl format > file description which will do something sane (not just ignore) fields > which are annotations added by anno ? I'm not sure what you consider sane. They can be combined, but mhl can only be told a field contains addresses or dates, not both. And combining doesn't strip duplicates, e.g. ‘a@b.c’ below. $ cat 1 Date: Sat, 14 May 2022 11:13:58 +0100 foo: a@b.c, d@e.f foo: Sat, 14 May 2022 10:13:58 +0100 foo: a@b.c, h@h.i foo: Sat, 14 May 2022 11:13:58 +0100 bar: x@y.z bar: Sat, 14 May 2022 11:13:59 +0100 $ $ cat format :Before foo:compress,nosplit,overflowoffset=4 bar:compress,nosplit,overflowoffset=4 :After $ $ /usr/lib/nmh/mhl -w 72 -form format 1 Before foo: a@b.c, d@e.f Sat, 14 May 2022 10:13:58 +0100 a@b.c, h@h.i Sat, 14 May 2022 11:13:58 +0100 bar: x@y.z Sat, 14 May 2022 11:13:59 +0100 After $ > About their only other defining feature is that they precede all other > fields in the message (even the return-path field). If anno(1)'s -append isn't given. -- Cheers, Ralph.
Re: new release?
Hi David, > I rely on the exit status of build_nmh, and only look at the log file > if indicates failure. Ah, I was considering any tests skipped as failure because I was trying to get them all to run. :-) I agree in the general case that's not wanted. > Should we output the summary to the terminal with -v? +1 -- Cheers, Ralph.
Re: new release?
Hi David, > > > Both releases saw a spew of "Broken pipe" errors after > > > test-mhparam, not sure if that's expected or not. > > > > That's not expected. The test immediately after that test-mhparam > > is test/oauth/test-send. So Jay's later email showed EPIPE was being returned and that isn't expected in test/server.c's putcrlf(). It could easily be added to the other expected errno values, but I wonder if none of those errno values should be there and instead one of our clients is closing the connection rather than reading to the end of what's being sent? -- Cheers, Ralph.
Re: new release?
Hi Eric, > I've pushed f188de902b613110d5943210f212adb680a2b44a to master > now, which should resolve this test failure. Thanks, but could you, and any other committers, try to use Git's rebase rather than merge as it preserves your commits and their messages rather than ‘Merge commit 'f188de902b613110d5943210f212adb680a2b44a'’ which doesn't spell out the fix for those browsing the commit log. ‘git pull --rebase’ does it, and that option can be made the default. As well as git-pull(1), there's an explanation in https://git-scm.com/book/en/v2/Git-Branching-Rebasing -- Cheers, Ralph.
Re: new release?
Hi again David, > Installing libgdbm-compat-dev for ndbm.h works, i.e. ./build_nmh > installing to /tmp and leaving the rest to configure ends > > All 99 tests passed > (19 tests were not run) Sometimes, the summary of the tests run by build_nmh do not appear on the terminal. $ ./build_nmh Install prefix [/usr/local/nmh]: /tmp/nmh Locking type (dot|fcntl|flock|lockf) [determined by configure]: MTS (smtp|sendmail/smtp|sendmail/pipe) [smtp]: SMTP server [localhost]: Cyrus SASL support (y|n) [determined by configure]: TLS support (y|n) [determined by configure]: $ They're tacked onto the end of the log file, as if the restoration of stdout failed. $ tail -4 build_nmh.log | sed -n l ===$ \033[0;32mAll 100 tests passed\033[m$ \033[0;32m(18 tests were not run)\033[m$ ===$ $ -- Cheers, Ralph.
Re: new release?
Hi Paul, > At first I was missing "yacc". Installing either bison or byacc gets > me to this error. Has configure been re-run since the installation? > $ make > make all-am > make[1]: Entering directory '/home/pgf/src/pdom/nmh/nmh.git' > /bin/bash ./ylwrap sbr/icalendar.l .c sbr/icalendar.c -- : > make[1]: *** [Makefile:4673: sbr/icalendar.c] Error 1 > make[1]: Leaving directory '/home/pgf/src/pdom/nmh/nmh.git' > make: *** [Makefile:1902: all] Error 2 Here's how make runs it for me. /bin/sh ./ylwrap sbr/icalendar.l lex.yy.c sbr/icalendar.c -- flex So your .c should be lex.yy.c or similar depending what your lex(1) produces. The argument are Usage: ylwrap [--help|--version] INPUT [OUTPUT DESIRED]... -- \ PROGRAM [ARGS]... Wrapper for lex/yacc invocations, renaming files as desired. INPUT is the input file OUTPUT is one file PROG generates DESIRED is the file we actually want instead of OUTPUT PROGRAM is program to run ARGS are passed to PROG Any number of OUTPUT,DESIRED pairs may be used. -- Cheers, Ralph.
Re: new release?
Hi David,
A bit of quick feedback on Debian 11...
MACHINES mentions:
gdbm-devel, db4-devel or libdb-devel/libdb-dev (only needed for
slocal(1))
There's no db4-devel. A libdb5.3-dev exists and is installed but it
doesn't satisfy configure's search.
libdb-dev is installed but it doesn't satisfy configure's search.
checking for dbm in ndbm.h... no
checking for dbm in db.h and db... no
checking for dbm in ndbm.h and db... no
checking for dbm in ndbm.h and db1... no
checking for dbm in ndbm.h and ndbm... no
checking for dbm in db1/ndbm.h and db1... no
checking for dbm in gdbm/ndbm.h and gdbm... no
checking for dbm in gdbm/ndbm.h and gdbm_compat -lgdbm... no
checking for dbm in ndbm.h and gdbm... no
checking for dbm in ndbm.h and gdbm_compat -lgdbm... no
checking for dbm in gdbm-ndbm.h and gdbm_compat... no
configure: error: could not find a working ndbm library/header combination
see nmh MACHINES file for build dependences
There's no gdbm-devel, but there is libgdbm-dev and installing that
still didn't help configure; I got the same output as above.
db.h is present, but the other test above still fails, presumably
correctly because it's changed too much.
libdb5.3-dev: /usr/include/db.h
Ah, ndbm.h is in
libgdbm-compat-dev: /usr/include/gdbm-ndbm.h
libgdbm-compat-dev: /usr/include/ndbm.h
Installing libgdbm-compat-dev for ndbm.h works, i.e. ./build_nmh
installing to /tmp and leaving the rest to configure ends
All 99 tests passed
(19 tests were not run)
A separate ‘make check’ shows lots of tests ‘skipped: no suitable locale
available’.
$ locale -a
C
C.UTF-8
en_GB.utf8
POSIX
$
$ git grep require_locale
test/common.sh.in:require_locale ()
test/dist/test-dist:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/format/test-ap:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhbuild/test-attach:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhbuild/test-cte:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhbuild/test-ext-params:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
test/mhbuild/test-header-encode:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
test/mhbuild/test-utf8-body:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
test/mhfixmsg/test-mhfixmsg:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
test/mhical/test-mhical:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhl/test-format:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhl/test-rfc6532:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhlist/test-ext-params:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
test/mhshow/test-charset:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/mhshow/test-textcharset:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
test/pick/test-pick:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/post/test-rfc6531:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/repl/test-convert:require_locale en_US.UTF-8 en_US.UTF8 en_US.utf-8
en_US.utf8
test/scan/test-scan-multibyte:require_locale en_US.UTF-8 en_US.UTF8
en_US.utf-8 en_US.utf8
$
I see require_locale treats its argument as a grep(1) regexp but the
dots aren't escaped.
# Skip test if none of the offered locales are supported.
# As side effect, use the first locale that is found. Note that
# some platforms allow, by way of example, en_US.UTF-8 to be used
# even though en_US.UTF8 is listed by locale -a. But by setting
# LC_ALL here, we don't rely on that.
require_locale ()
{
for locale in "$@"; do
if locale -a | grep -i "$locale" >/dev/null; then
LC_ALL="$locale"; export LC_ALL
return
fi
done
test_skip "no suitable locale available"
}
Perhaps, if the tests would still work, it should accept regexps, filter
‘locale -a’ output with each in turn, as it does now, but sort the
output in the C locale and use the first. Then the calls could be
require_locale '^en_..\.[Uu][Tt][Ff]-*8$'
As a quick test of a non-US locale, altering require_locale() to
require_locale ()
{
+LC_ALL=en_GB.utf8; export LC_ALL
+return
+
got most locale-using tests to pass. test/mhbuild/test-utf8-body still
wasn't happy because it hard-codes ‘en_US.UTF-8’. Using LC_ALL set by
require_locale() instead had the test pass.
diff --git a/test/mhbuild/test-utf8-body b/test/mhbuild/test-utf8-body
index e3903ae6..2022e98f 100755
--- a/test/mhbuild/test-utf8-body
+++ b/test/mhbuild/test-utf8-body
@@ -174,6 +174,7 @@ Nmh-Attach: $MH_TEST_DIR/attachment.txt
¡Ay, caramba!
EOF
+found_locale=$LC_ALL
Re: new release?
Hi Howard, > Not a git user, so the problem could be on my end as I'm not getting a > version that matches 1.7.1 (or newer). ... > git clone -b master git://git.savannah.nongnu.org/nmh.git That looks right to me. You've asked for the ‘master’ branch where development takes place. To show branches with the current one highlighted: git branch To describe the commit which is checked out: git describe --long --tags --candidates=1000 -- Cheers, Ralph.
Re: merge pick and scan
Hi Philipp, > > Yes, if pick always listed by default then it could skip the writing > > of sequences by passing them as arguments to mark. But bear in mind > > the limit on the number of arguments to a process and their total > > size in characters, especially some decades ago. ... > So is this limitation still relevant? And are there other solutions > like let mark read from stdin or use xargs? The xargs(1) route is tedious and not what you'd do on the fly. I suppose mark(1) reading from stdin would be an option. I have a script which maintains several sequences as I interactively move through emails and I suffer a noticeable delay on this old machine due to calling mark(1) several times to update a different sequence each time. Really, I'd like to run mark once giving it a list of what to do per sequence for a number of sequences. Then it can just read and write the sequences file once, speeding things up. The standard input route wouldn't work for that if it were just a simple list of message numbers. > > > To get these you can currently choose between "pick seq", > > > > Only with ‘-list’ which you may have as a default in your > > ~/.mh_profile. > > Just a litle nitpick: "-list" is the default if no sequence is given. Ah, then it's me that has a default: a -sequence for pick in my profile! > > With the Unix ‘do one thing well’ outlook, > > > > - pick's single purpose is to search emails with its ‘matcher’ > > operators. > > The point where we disagree is what should pick do with the matched > emails. Well, no... > > - scan's single purpose is a one-line summary of a message, perhaps > > just the message number. You think pick should scan the matched emails, thereby merging two one-things, pick and scan, into one. This is no longer ‘do one thing well’. I agree that means we disagree what pick should do with matched emails, but clouds the reason. > > Moving the ‘matcher’ arguments from pick to all the other commands > > would improve the UI and they'd be consistent in that ‘-subject’ > > could be given to all of them. > > Then there would be no tool to match/search emails. Correct. Just as no tool is needed to list a sequence's message numbers, instead ‘show seqname’ works. > Yes, with my suggestion there would be no tool to display a one-line > summary. Agreed. But I think that's a different operation to searching emails and as long as searching is a distinct action, as opposed to all commands accepting matcher arguments, then I think it should be a distinct command. > Also as said earlier this would require all tools to be able read and > parse the mails. Yes this would be hidden in srb/, but still be there. > So all tools would match and do there purpose. Yes, but it's no extra work. It's less because two commands and passing data between them, either with pick's -list into argv[] or through .mh_sequences, would not be needed. > > Moving the ‘matcher’ arguments to just scan would be a wart. Yes, I > > know you think of it as scan's one-line formatting moving to pick, > > but it's the same thing looked at the wrong way around simply > > because of how you see to implement it. > > I see it this way, because of the way I use it. Most of the time I use > the matchers of pick, I search for the next mail to handle and > therefor use a combination of scan and pick. I don't use sequences > that much. Ah, I pick and then move through a sequence. ‘n p’ shows me the next in sequence p, pick's default for me, and ‘b p’ goes back. (Not ‘p’ for prev(1)-ious because ‘p’ for me is pick(1). :-) > > pick and scan overlap, but that's due to the error of adding -list > > to pick and isn't a reason to merge the two. > > I still see the error in pick in the -sequence switch. Without sequences, all commands would need to accept message numbers on stdin to work around argv[] limits which would hamper stdin's use for other, more interesting, purposes. > > Your suggestion is one interpretation and I don't think it's the > > best one. Anointing your suggestion by making your change would > > further muddle the two distinct operations of scan and pick. > > I think here we are at a point we can agree to disagree. Yes. :-) -- Cheers, Ralph.
Re: merge pick and scan
Hi Philipp, > Ralph wrote: > > > 1. selected messages from a folder 2. unselect messages with don't > > > match the given pattern 3. Print out all selected messages > > > > I don't think pick(1) does item 2, unless you mean by virtual > > [virtue] of item 1 leaving some messages out. > > Yes I mean this more virtual, the implementation details are not realy > importend for my point. > > > And I think it shouldn't do item 3. When sequences were added to MH > > and pick altered to define them, scan(1) could have been used to get > > the message numbers, similarly to how you suggested earlier. > > Ok I see this more the other way around, pick shoundn't write > sequences, there is mark(1) for this. Yes, if pick always listed by default then it could skip the writing of sequences by passing them as arguments to mark. But bear in mind the limit on the number of arguments to a process and their total size in characters, especially some decades ago. It would have been annoying for a search with pick which produced many messages to be awkward to store in a sequence by requiring multiple runs of mark, building up the sequence piecemeal. Or for refile `pick ...` +foo to not be run by the shell because too many messages were matched. Thus sequences were created to store pick's results and all commands which accepted message numbers altered to also accept sequence names. One could argue, not that I am, that pick should have always listed the message numbers and sequences need not exist, but limits of the day prevented that. > I don't know whitch suggestion you mean. I guess you mean something > to get the message numbers of a sequence. Yes. > To get these you can currently choose between "pick seq", Only with ‘-list’ which you may have as a default in your ~/.mh_profile. > "scan -format '%(msg)' seq" That's the one I was thinking of. > and "mark -list -sequence seq". The last one prints the contet of > .mh_sequences and is special for sequence management. mark's output isn't suitable as a list of message numbers to pass to other commands, even with ‘-norange’. > But the scan and pick version do more or less the same. Yes. > > > Scan: > > > > > > 1. select messages from a folder > > > 2. Print out all selected messages > > > > > > The function of the two tools looks overlapping. > > > > Agreed, but compounding the wrongs doesn't make a right. :-) > > Are they realy that wrong or is this a question of the perspective? Oh, this is subjective to an extent. > From my point of view pick does message selection and printing. The > sequence handling is kind of an odd feature. > > If I understand you correct you see pick more as a tool to add > messages to a sequence and the -list switch is kind of odd. Yes, pick was created to search for messages to store in a sequence, and sequences created at the same time. pick's ‘-list’ duplicates scan(1)'s functionality. pick taking message numbers and no ‘matcher’ arguments duplicates mark's functionality. > So in both views pick can do something which don't realy match. Also > there are other tools (scan and mark) with can do parts of the > functions better. So it's reasonable to ask what is the purpose of > pick. My suggestion would clarify the purpose. Your suggestion is one interpretation and I don't think it's the best one. Anointing your suggestion by making your change would further muddle the two distinct operations of scan and pick. > > > If this would be added to pick, then scan would be obsolet. > > > > If all commands grew message-set expressions, like pick's, then pick > > would be obsolete. ;-) mark(1) would do when the sequence only > > needs modifying without display. > > Yes, but would it be reasable to add message-set expressions to all > tools? As you pointed out they where removed some time ago. No, they weren't. I didn't point that out. They have only ever been arguments to pick. > This would also require all tools to read and parse the messages. > Currently not all tools (i.e. rmm) need to do this. If ‘-subject foo’ were used when calling rmm then yes, it would need to read messages, but ‘rmm 42 314’ wouldn't bother to read 42 or 314. The processing of the ‘matcher’ arguments, like ‘-subject’, would be done by library code called by all the commands which accepted them. Just as common code now allows a sequence name to be used in place of message numbers. > But looking at pick: pick already reads and (somehow) parses the > messages. A -form/-format switch would only add a way to print this > information. It would compound the errors already made to pick. pick -seq foo 42 314 # Should be mark(1). pick -seq foo -list -subject foo # Should use scan(1) afterwards. > With leads to the question: Why does pick only print out message > numbers and not a mh-format(5). The obvious awnser is: Because there > is scan for this. But you could also ask: Is scan (as an extra tool) > required at all? A
Re: merge pick and scan
Hi Conrad, > but here's a case that is currently unambiguous (because MH commands > have side effects) that might cause complications for general switch > parsing: > > refile +foo `pick ... +bar` I agree. The search folder would have to be set first as refile's folder would always be the destination to remain backwards compatible. folder +bar refile +foo -from foo > Indeed, why wouldn't we then want to say > > refile -from john +* +foo At some point, the syntax can't handle what ever's required and multiple commands would be necessary. But given ( and ) should be synonyms for -lbrace and -rbrace anyway, borrowing from find(1), perhaps refile +foo \( -from john ++\* \) # Like bash's globstar. But I have been thinking about sequences which span folders. Is a sequence really just an optimisation of a folder containing linked messages? Except that an empty sequence is, wronly IMO, not supported. This opens the path to all messages in a single store with folders/sequences being ‘views’ onto those messages. Then do folders move to an implementation more like sequences? With a ‘synchronise export’ operation to bring a directory tree up to date with a sequence/folder. -- Cheers, Ralph.
Re: merge pick and scan
Hi again Philipp, > David wrote: > > I don't think that departure from the Unix philosophy can be > > considered good for nmh. > > I don't quit understand this argument. From my perspective the current > interface don't match the Unix philosophy. To explain this lets look > what scan and pick do: > > Pick: > > 1. selected messages from a folder > 2. unselect messages with don't match the given pattern > 3. Print out all selected messages I don't think pick(1) does item 2, unless you mean by virtual of item 1 leaving some messages out. And I think it shouldn't do item 3. When sequences were added to MH and pick altered to define them, scan(1) could have been used to get the message numbers, similarly to how you suggested earlier. > Scan: > > 1. select messages from a folder > 2. Print out all selected messages > > The function of the two tools looks overlapping. Agreed, but compounding the wrongs doesn't make a right. :-) > If this would be added to pick, then scan would be obsolet. If all commands grew message-set expressions, like pick's, then pick would be obsolete. ;-) mark(1) would do when the sequence only needs modifying without display. Now, you could ask how can I support the Unix Philosophy of ‘do one thing well’ and that pick should not have -list but that all commands should grow ‘-from foo’? Well, it's the demarcation of that one thing. ‘grep -l’ was good enough before pick came along and when better functionality was needed it was presumably easiest to concentrate the new experimental syntax in a new command rather than come up with a syntax which didn't conflict with all the different commands existing arguments. With hindsight, and decades of improvement in computing power, I think a general syntax for describing sets of messages, including the empty set, available to all nmh commands which manipulate messages would be more orthogonal. Mercurial's ‘revset’ notation is an example of the kind of thing, e.g. to remove all messages from foo and any replies to them and replies to replies ad infinitum by making use of the threading fields: rmm 'descend(from:foo)' > > There needs to be really good justification for intentional > > breakage. > > I hate this argument. Yes changing the user interface shouldn't be > done just because it feels good. But To improve the user interface it > must change sometimes. But changing the user interface doesn't have to cause breakage. -- Cheers, Ralph.
Re: merge pick and scan
Hi Philipp, > To scan selected messages currently a combination of scan and pick is > used. It would be nice to be able to write something like: > > scan -from bob > > In order to do this we could add the -form/-format switches to pick > and make scan a link to pick. To avoid most breakage pick would use > "%(msg)" format by default and scan.default if called as scan. This > way only some scan aliases would break (if -form/-format is not > explicitly set for them). All other usages would still work. Did you know pick(1) says HISTORY In previous versions of MH, the pick command would show, scan, or refile the selected messages. This was rather “inverted logic” from the Unix point of view, so pick was changed to define sequences and output those sequences. Hence, pick can be used to generate the arguments for all other MH commands, instead of giving pick endless switches for invoking those commands itself. I'm generally not in favour because it seems to be combining two specific things rather than thinking more orthoganally. For example, I'd quite like to do refile +archive -from foo -and -subject bar forw -mime -date '01 Apr' IOW, make pick's expressions first-class citizens of all nmh command parsing. Except I think the syntax is wordy and something like refile +archive from:foo subject:bar would be better, including the implicit logical conjunction. -- Cheers, Ralph.
Re: mhfixmsg character set conversion
Hi Steven, > > To be fair ... that's completely permitted according to the spec! > > That doesn't make it a good idea. :-) Well, I think it is a good idea and that's probably why it was permitted. I don't recall precisely, and don't have time to check, but I think the RFC even recommends it. This is because what can be plain ASCII stays plain ASCII and you just encode the odd bit which can't. This makes it easier for humans reading the still-encoding header field. > The goal is, and has always been, to save the message in a format that > can be searched easily at a later date. I'm rusty, but my recollection is things like ‘forw -mime’ and dist(1) assume ~/mail/inbox/42 is in a valid format to ship and don't do any checks or conversions. mhfixmsg(1) repeatedly mentions RFC violations in some of its possible outputs. So having searched and found it, don't send it on. :-) -- Cheers, Ralph.
Re: Problem with mhl.format
Hi Arthur, > Now if I run show on a mail message in +inbox the x-spam-report does > not show. However if I run show on a message in +nmh the x-spam-report > does show along with content-type, message-id and every other header > field. Picking on the simplest problem first, I'm not aware of this behaviour being per folder so I can't explain that unless the +inbox message doesn't have a x-spam-report field whereas the +nmh one does. With the current message in each folder set appropriately, what does this give? egrep -Him1 '^$|x-spam-report:' `mhpath +inbox .` anno -list -number -component x-spam-report +inbox . egrep -Him1 '^$|x-spam-report:' `mhpath +nmh .` anno -list -number -component x-spam-report +nmh . -- Cheers, Ralph.
