Re: nmh 1.8RC2, xlbiff, and $HOME
Ralph wrote: > Your principles soon went out the window. Oh no, they're all still there. One of them just got moved to the front. David
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, > Then I think that we should do the same in 1.8 for all platforms. Your principles soon went out the window. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
az wrote: > minor data point: after following the discussion here, i've decided > that for now _the debian packages_ of nmh should not distinguish > between unset $HOME and set-but-empty $HOME, and that either should cause a > fallback to getpw*. Then I think that we should do the same in 1.8 for all platforms. Thoughts on applying that patch (plus a test and documentation update) to create a 1.8RC3 this weekend? David
Re: nmh 1.8RC2, xlbiff, and $HOME
On Thu, 02 Feb 2023 16:44:05 -0800, David Levine writes: >Anyways, I'd like to keep the 1.7 >behavior for 1.8 and change it after. Can we agree to that? minor data point: after following the discussion here, i've decided that for now _the debian packages_ of nmh should not distinguish between unset $HOME and set-but-empty $HOME, and that either should cause a fallback to getpw*. (1.8RC2 will hopefully/likely be part of the upcoming debian release; the previous release shipped with 1.7.1.) regards az Description: treat set-but-empty $HOME same as unset upstream code aborts on empty $HOME and falls back to getpw* only on unset $HOME, which is undesirable (affects xlbiff's testsuite and generally surprises end users). this patch makes it fall back for both empty and unset $HOME. Author: Alexander Zangerl Bug: https://bugs.debian.org/1029752 Last-Update: 2023-02-02 --- a/sbr/path.c +++ b/sbr/path.c @@ -38,10 +38,7 @@ void set_mypath(void) { char *var = getenv("HOME"); -if (var) { -if (!*var) -die("environment variable HOME is empty"); - +if (var && *var) { mypath = mh_xstrdup(var); return; } -- Alexander Zangerl + GPG Key 2FCCF66BB963BD5F + https://snafu.priv.at/ He who joyfully marches to music in rank and file has already earned my contempt. He has been given a large brain by mistake, since for him the spinal cord would fully suffice. -- Einstein signature.asc Description: Digital Signature
Re: nmh 1.8RC2, xlbiff, and $HOME
I wrote: > What case am I missing? I get it: the non-null, empty HOME. Anyways, I'd like to keep the 1.7 behavior for 1.8 and change it after. Can we agree to that? David
Re: nmh 1.8RC2, xlbiff, and $HOME
Ralph wrote: > Hi David, > > > And my feeling at this point is to put out an RC3 with the 1.7 > > behavior for $HOME. > > Which, to be clear, is not the behaviour Ken and kre want. :-) I was thinking that just removing the if (!*var) statement from your code, leaving: char *var = getenv("HOME"); if (var) { mypath = mh_xstrdup(var); return; } errno = 0; struct passwd *pw = getpwuid(getuid()); if (!pw) { if (errno) adios("", "getpwuid() failed"); /* "" prints errno! */ die("password entry not found"); } if (!*pw->pw_dir) die("password entry has empty home directory"); mypath = mh_xstrdup(pw->pw_dir); is the same as the 1.7 code (ignoring the string copies): if ((mypath = getenv("HOME")) == NULL) { if ((pw = getpwuid(getuid())) == NULL || *pw->pw_dir == '\0') adios(NULL, "cannot determine your home directory"); else mypath = pw->pw_dir; } What case am I missing? David
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, I've had one night's sleep spread over the last two nights so I'm not going to have time for nmh until the weekend. > And my feeling at this point is to put out an RC3 with the 1.7 > behavior for $HOME. Which, to be clear, is not the behaviour Ken and kre want. :-) Stephen, I'd appreciate knowing if unsetting HOME with RC2, e.g. ‘env -u HOME ...’, is sufficient for xlbiff's test harness. $ HOME= MH=$PWD/mh_profile mhparam path mhparam: environment variable HOME is empty $ $ env -u HOME MH=$PWD/mh_profile mhparam path /tmp -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Ralph wrote: > The behaviour cannot solidify further. :-) It's clear, determinate, > predictable, and simple to document. Those qualities can be true for the 1.7 behaviour. > I thought RCn+1 was meant to be minimal differences from RCn? If there's an RC3, the only code difference would be for this issue. > That's why Andy Bradford's POP3 patch is post 1.8. There's a key difference between the $HOME change and Andy's POP3 patch: the behaviour has already, prior to 1.8 final release, changed from 1.7 for the former but not the latter. > Switching the behaviour of > a set-but-empty HOME now seems significant given some value for HOME > must be chosen. 1.8 hasn't been released yet. I think that we owe users consistent behaviour from 1.7 to 1.8 unless clearly documented. > And will mean another round of testing by others on top of your time. I appreciate your consideration of that but I think that we need to resolve the consistency issue. In any case, a new test and man page update are called for. And my feeling at this point is to put out an RC3 with the 1.7 behavior for $HOME. David
Re: nmh 1.8RC2, xlbiff, and $HOME
Date:Wed, 01 Feb 2023 14:54:36 + From:Ralph Corderoy Message-ID: <20230201145436.1aa5222...@orac.inputplus.co.uk> | My stance has nothing to do with satisfying âuser expectationsâ so I'll | ignore this aspect. Ralph, do you not see the contradiction between that, and: | If 3 is implemented then the user may have thought a different pretence | would occur. which is exactly a desire to satisfy user expectations, and when you don't know what they are, refuse to do anything. There is plenty in nmh that doesn't act (by default) to satisfy my expectations, but that's OK, it works as documented, and there is generally a method by which I can achieve what I want. Why does this need to be different? | After all, it's an arbitrary choice. Perhaps he expects it | will carry on just like his shell does. Perhaps. But we cannot meet that objective, as shells do different things, there are many to choose from, but there is just one nmh (well two, if you count the mmh I recall hearing of). Do you think we should create nmh forks so users can pick the one that meets all their expectations by default? | This could result in surprising files being read, e.g. /tmp/.mh_profile, Yes, if someone's pw_dir is /tmp ... but then that is what it should do. No-one is suggesting treating an empty HOME literally (as "") in this context, nor as "." or anything odd - just treat it the same as an unset HOME. If the user doesn't like that, then HOME should be set to whatever they want it to be. Simple. Clean. Easy to document. | mh_profile(5) can easily say HOME must be non-empty if set for those | that read the fine manual. It can. But what's the point? | No, you must decide which of the many alternatives of 3 above is the | ânmh wayâ, document that, test it, and have some users be surprised by | the result. We have already had users (the xlbiff test suite) be surprised by the current result - which was not only unexpected, it varied from earlier behaviour. | Ken, you, and David all seem irked by unset and set-but-empty being | treated differently, as if you'd like a binary outcome by first | conflating the ternary input to binary. There isn't a clear method of | merging those two inputs. Ternary's messy, as ever. We can't stop HOME | being unset or being empty so there are three safe outcomes for three | inputs. That's solely because that's the way you see it. It is entirely possible to treat two different settings (such as +0 and -0 in one's complement arithmetic) as meaning the same thing in one context, such as they do when using normal + - * / operations, yet a different one in a different context (like & | ^ bit operations). That's just the way things work. Treating an unset HOME and an empty but set HOME the same in nmh is a perfectly valid choice, and seems to be the preferred one. kre
Re: nmh 1.8RC2, xlbiff, and $HOME
>Ken, you, and David all seem irked by unset and set-but-empty being >treated differently, as if you'd like a binary outcome by first >conflating the ternary input to binary. Sigh. I'll try to make my point clearer, but I recognize we're not going to agree on this. Since David is driving this release, I defer to his decision in terms of changing the behavior for the next release candidate (I would vote for making the behavior of unset and empty be the same, but I consider David the God Emperor of nmh 1.8, so it's up to him). You ask the question: What is the user trying to achieve by explicitly setting an empty HOME? To me, the answer is obvious: they were trying to create a clean environment. You say that they could do that by unsetting HOME; well, count me in the pool of people who did not know until now that 'unset' even existed. I mean, I vaguely knew that there was a difference between a variable that was never used and one that was set to an empty string, but as far as I could tell there wasn't a PRACTICAL difference between those two states and I certainly wouldn't expect an application to treat those states differently. I suspect the person who wrote that test for xlbiff didn't know about 'unset' either and would be bewildered that there is any difference in behavior. Also, some quick testing suggests that: % HOME='' command Does the expected thing but: % unset HOME command Simply unsets the HOME and 'command' variables. I realize you could do that in a subshell but it just seems awkward, and the idiom of doing % FOO=value command to change an environment variable is very common and I see why people would just use that. I have read your arguments carefully and ... well, I still do not agree that there should be a difference in behavior between an unset HOME and an empty HOME. To me there is no ambiguity and they are the same, even if it's a technically different variable state. Again, I realize we're just not going to agree on this. --Ken
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi, I wrote: > If 3 is implemented then the user may have thought a different > pretence would occur. After all, it's an arbitrary choice. Perhaps > he expects it will carry on just like his shell does. This has overlap with DWIM, ‘do what I mean’, and Perl's TMTOWTDI, ‘there's more than one way to do it’. DWIM obviously gets one tied to chair. http://www.jargon.net/jargonfile/d/DWIM.html I'm a fan of Perl, programming a lot of it in Perl 4 and early 5 days, but having introduced it to a large site of engineers who did a bit of programming, I came to see TMTOWTDI as a problem. I could read all their code because I knew all the ways. They wrote in dialects by using different subsets, reducing re-use and understanding. Many saw TMTOWTDI as a problem. It's one reason why Python has boomed. Amongst its ‘import this’ Zen are Explicit is better than implicit. In the face of ambiguity, refuse the temptation to guess. There should be one-- and preferably only one --obvious way to do it. An empty HOME isn't explicit in its meaning, it's ambiguous, we shouldn't guess, and the obvious way of specifying the home directory is to set HOME to it. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, > > This close to a release, I think we should stick with requiring HOME > > to be non-empty if it's set as otherwise there's too many paths to > > consider which the test harness probably doesn't exercise. > > I'd rather crank out an RC3 than pass up the opportunity to solidify > the behaviour here. The behaviour cannot solidify further. :-) It's clear, determinate, predictable, and simple to document. I thought RCn+1 was meant to be minimal differences from RCn? That's why Andy Bradford's POP3 patch is post 1.8. Switching the behaviour of a set-but-empty HOME now seems significant given some value for HOME must be chosen. And will mean another round of testing by others on top of your time. > Ken's point that different treatmet of unset vs empty HOME is a mistake > is fairly compelling for me. I've covered that in my reply to kre just now. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi kre, > There is never a need to deliberately make anything an error to > satisfy user expectations, if the user wants a complaint about HOME > being an empty string, rather than some other behaviour, all they need > is one of... My stance has nothing to do with satisfying ‘user expectations’ so I'll ignore this aspect. > > No, one cannot say of an unset HOME that it may be set by accident. > > No, but it may have been unset by accident, where the intended value > for mh to use was something different than the pw_dir entry. No-one > can possibly know in that case, any more than in the case where > HOME='' (I think you've over-cropped in general.) There is a marked difference which I thought I had made very clear. There are three cases. All may be accidental or deliberate, obviously. 1. - HOME is unset. - This is valid, i.e. POSIX allows it. - The only route for continuing is it to use a mandatory passwd.pw_dir. - HOME=passwd.pw_dir results. 2. - HOME is set and non-empty. - Use it. - If that causes errors then die, do not try again with passwd.pw_dir. - HOME=$HOME results. 3. - HOME is set but empty. - There are at least three ways of continuing. - Pretend HOME=$PWD - Pretend HOME=/ - Pretend HOME=passwd.pw_dir - Other alternatives include trying HOME=... until one ‘works’. - HOME=??? results. There is no obvious way to continue for 1 and 2 other than the above. The result is not surprising. Even if the manual isn't read. If 3 is implemented then the user may have thought a different pretence would occur. After all, it's an arbitrary choice. Perhaps he expects it will carry on just like his shell does. This could result in surprising files being read, e.g. /tmp/.mh_profile, and unwanted commands being run, e.g. through a profile's ‘moreproc’ or ‘mhshow-show-text/plain’. (top(1) had a similar issue. It used $PWD when HOME was unset or empty allowing a privilege escalation through its configuration file; cue CVE.) Turning it about. What is the user trying to achieve by explicitly setting an empty HOME? Whatever it is, he can achieve it in an *unambiguous* manner by unsetting HOME or setting it to a non-empty value. It seems you're arguing for him to instead first refer to the manual to determine what set-but-empty means to nmh in the hope it matches his need when he could just take and explicit short-cut? > There is always ambiguity if you are doing something different than > what is documented to happen. That is not ambiguity. That is error. mh_profile(5) can easily say HOME must be non-empty if set for those that read the fine manual. > The question is what is best for nmh - and for that I agree with Ken, > treating an unset HOME and HOME='' differently is a mistake. > All that is required to make that OK is to document it. No, you must decide which of the many alternatives of 3 above is the ‘nmh way’, document that, test it, and have some users be surprised by the result. There is no ‘All that...’. Ken, you, and David all seem irked by unset and set-but-empty being treated differently, as if you'd like a binary outcome by first conflating the ternary input to binary. There isn't a clear method of merging those two inputs. Ternary's messy, as ever. We can't stop HOME being unset or being empty so there are three safe outcomes for three inputs. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Ralph wrote: > This close to a release, I think we should stick with requiring HOME to > be non-empty if it's set as otherwise there's too many paths to consider > which the test harness probably doesn't exercise. I'd rather crank out an RC3 than pass up the opportunity to solidify the behaviour here. Ken's point that different treatmet of unset vs empty HOME is a mistake is fairly compelling for me. David
Re: nmh 1.8RC2, xlbiff, and $HOME
Ralph Corderoy wrote in <20230131181958.1cfb121...@orac.inputplus.co.uk>: ... |But if HOME is empty we do not know their intent so to ignore it and use |pw_dir may not be what they think will occur. The wrong profile could |be read or the wrong .netrc used, upsetting the user. By the way my mailer does: ? xv var HOME #nodelete,import-environ-first,sync-environ,notempty: set HOME=/home/steffen (where xv temporarily enables *verbose* mode). If i recall correctly you, once you were still taking part, have been dissatisfied with TOCTOU issues for TMPDIR for which the same holds: ? xv var TMPDIR #default-value,import-environ-first,sync-environ,notempty: set TMPDIR=/tmp So i (fwiw) are in the camp $ HOME= s-nail -R# s-nail: Environment variable must not be empty: HOME s-nail: There are messages in the error ring, manageable via `errors' command #?0!0/NONE#ERROR|:/dev/null? var HOME set HOME=/home/steffen #?0!0/NONE#ERROR|:/dev/null? xit --steffen | |Der Kragenbaer,The moon bear, |der holt sich munter he cheerfully and one by one |einen nach dem anderen runter wa.ks himself off |(By Robert Gernhardt)
Re: nmh 1.8RC2, xlbiff, and $HOME
Date:Tue, 31 Jan 2023 13:29:46 + From:Ralph Corderoy Message-ID: <20230131132946.44a5f20...@orac.inputplus.co.uk> | > Similarly, in XCU 4 in the description of the cd utility: | Yes, but also allowed there is âemptyâ which also triggers | implementation-defined. Yes, that's because of: | Also, POSIX's chdir(3) says |int chdir(const char *path); | ERRORS |The chdir() function shall fail if: |... |ENOENT A component of path does not name an existing |directory or path is an empty string. and since some shells simply do effectively chdir(getenv(HOME)); in this case, they get an error, if HOME is unset, or empty. Others check the value from getenv(HOME) and do something different if it is NULL, and still others go further and check for "" as the value, and either treat that the same as a NULL result, or in some other way that suits them. Note that POSIX documents what users can expect to have happen (what can be relied upon) and only rarely mandates some specific behaviour that the people who maintain it believe should happen, rather than what actually does. If HOME is unset or empty, there is no consistent behaviour from shells, so POSIX says that. In the case of ~ expansion, there is no system call involved (necessarily) just string manipulation, so the case of HOME='' isn't treated specially (though it may end up being in the next version of the standard, as this is a case where there are some implementation differences - or perhaps it might be decided that the implementations that don't do what the standard currently requires are simply buggy, and leave things as they are now). There is (as of yesterday) a POSIX bug report about this, and some other weird case issues with ~ expansions. | The possible errors are now | environment variable HOME is empty The current question is why that one in particular needs to be an error. kre
Re: nmh 1.8RC2, xlbiff, and $HOME
Date:Tue, 31 Jan 2023 13:22:19 + From:Ralph Corderoy Message-ID: <20230131132219.5e02b20...@orac.inputplus.co.uk> | It looks to me like code assumes mypath isn't NULL, e.g. exmaildir(), | so not bothering to call set_mypath() if MH is set doesn't look a goer. Perhape not, but void set_mypath(void) { char *var = getenv("HOME"); if (var && *var != '\0') { mypath = mh_xstrdup(var); return; } errno = 0; struct passwd *pw = getpwuid(getuid()); /* etc */ would be a fairly trivial change (making the function smaller). It would also be possible to change if (!*pw->pw_dir) die("password entry has empty home directory"); to if (*pw->pw_dir == '\0') pw->pw_dir = "/"; /* or "/tmp" or even "/no/home/dir" or ... */ if that seemed desirable, kre
Re: nmh 1.8RC2, xlbiff, and $HOME
Date:Tue, 31 Jan 2023 18:19:58 + From:Ralph Corderoy Message-ID: <20230131181958.1cfb121...@orac.inputplus.co.uk> | No, one cannot say of an unset HOME that it may be set by accident. No, but it may have been unset by accident, where the intended value for mh to use was something different than the pw_dir entry. No-one can possibly know in that case, any more than in the case where HOME='' | There is no ambiguity over the right course. There is always ambiguity if you are doing something different than what is documented to happen. Users may have expected something different than you expect. | In the same way that seeing ${foo:-bar} instead of | ${foo:bar} in sh is a smell. Nonsense, ${foo:-bar} (rather than ${foo-bar} - a version with just a : would be a ksh, bash, or zsh extension - but in this case I assume just a typo)) is the more common thing to use. It matches the other common usage ${foo} which produces nothing when foo is unset, or when foo='' (and "${foo}" produces a null string in either of those cases). That a default value is to be used instead of the empty string doesn't change that at all - explicitly using ${foo-bar} is a much rarer use case (there are occasions when that's wanted, but they are not very common). There is no point comparing what shells do when HOME is unset or "", whether the same thing in the two cases, or something different (and seemingly bizarre implementations in unusual cases are often just because the cases are unusual - don't happen often - and slowing down the normal case to handle those specially isn't worth the effort). A shell can do whatever it likes in these cases. So can nmh. The question is what is best for nmh - and for that I agree with Ken, treating an unset HOME and HOME='' differently is a mistake. All that is required to make that OK is to document it. There is never a need to deliberately make anything an error to satisfy user expectations, if the user wants a complaint about HOME being an empty string, rather than some other behaviour, all they need is one of test -z "${HOME}" && { echo Error; exit ; } or test -z "${HOME-x}" && { echo Error; exit ; } (the latter being a case where omitting the ':' is wanted) My experience is that lots of users don't know "unset" exists in the shell. They just do var= (or unnecessarily var='') when then want a variable to "go away". Most of the time unset and empty variables mean the same thing, so this just works. kre
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi Ken, > > What's the intent of an empty HOME? > > Is it set by accident when it's meant to be unset? > > Is it empty by accident when it's meant to be non-empty? > > Do they want HOME=/, HOME=$PWD, or are they expecting it to error. > > Any choice could be not what the user intended so exit. > > I mean ... you could say the exact same things of an unset HOME! No, one cannot say of an unset HOME that it may be set by accident. Nor that it is empty by accident since it is neither empty or non-empty. This being Unix, the user knows what they're doing and POSIX seemingly allows it so we permit the lack of HOME and use pw_dir. There is no ambiguity over the right course. As is often the way with an environment variable, HOME overrides pw_dir just as an argv option overrides an environment variable which overrides a configuration-file setting. But if HOME is empty we do not know their intent so to ignore it and use pw_dir may not be what they think will occur. The wrong profile could be read or the wrong .netrc used, upsetting the user. > My point is that I cannot see a reason to treat unset HOME and empty > HOME differently in nmh; if unset HOME uses pw_dir, then I would argue > that an empty HOME should do the same thing. I disagree. In the same way that seeing ${foo:-bar} instead of ${foo:bar} in sh is a smell. > I was curious and some system call traces suggest that what happens > under the hood is that when 'cd' is called with an empty HOME sh > explicitly calls chdir(getcwd()). Sometimes. I think it's more complex than that, e.g. perhaps PWD has an effect, and it will obviously vary widely. $ LC_ALL=C env HOME= PWD=/nonexist strace -fe getcwd,stat,chdir dash -c cd stat("/nonexist", 0x7ffc101d1bb0) = -1 ENOENT (No such file or directory) getcwd("/home/tmp/1628693513.257008819", 4096) = 31 chdir("/home/tmp/1628693513.257008819") = 0 +++ exited with 0 +++ $ $ LC_ALL=C env HOME= PWD=$PWD strace -fe getcwd,stat,chdir dash -c cd stat("/home/tmp/1628693513.257008819", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 stat(".", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0 chdir("/home/tmp/1628693513.257008819") = 0 +++ exited with 0 +++ $ $ LC_ALL=C env -u PWD HOME= strace -fe getcwd,stat,chdir dash -c cd getcwd("/home/tmp/1628693513.257008819", 4096) = 31 chdir("/home/tmp/1628693513.257008819") = 0 +++ exited with 0 +++ $ -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
>What's the intent of an empty HOME? >Is it set by accident when it's meant to be unset? >Is it empty by accident when it's meant to be non-empty? >Do they want HOME=/, HOME=$PWD, or are they expecting it to error. >Any choice could be not what the user intended so exit. I mean ... you could say the exact same things of an unset HOME! My point is that I cannot see a reason to treat unset HOME and empty HOME differently in nmh; if unset HOME uses pw_dir, then I would argue that an empty HOME should do the same thing. Bakul says: >FWIW, this is how /bin/sh behaves on FreeBSD: >[...] Fair enough; I was curious and some system call traces suggest that what happens under the hood is that when 'cd' is called with an empty HOME sh explicitly calls chdir(getcwd()). But to me that still doesn't make a case that empty HOME and unset HOME should be treated differently in nmh. I recognize that this is a case where reasonable people can disagree. --Ken
Re: nmh 1.8RC2, xlbiff, and $HOME
On Jan 31, 2023, at 8:32 AM, Ralph Corderoy wrote: > > Hi Ken, > >>> So an unset HOME is allowed by this function, it's an empty HOME >>> which isn't. >> >> It strikes me as strange that there is a difference between an unset >> HOME and an empty HOME in terms of behavior. I mean, yes, I can see >> how the code is written, the historical precedent and how we got here, >> but ... well, I'm trying to understand the justification for treating >> those differently. > > A non-empty HOME is clear; its content should be used. > > An unset HOME is clear; it cannot be used. > There's a ready fallback, getpwuid(3) and pw_dir so use that. > > What's the intent of an empty HOME? > Is it set by accident when it's meant to be unset? > Is it empty by accident when it's meant to be non-empty? > Do they want HOME=/, HOME=$PWD, or are they expecting it to error. > Any choice could be not what the user intended so exit. FWIW, this is how /bin/sh behaves on FreeBSD: $ cd /tmp $ sh -c 'cd; echo $PWD' /home/bakul $ unset HOME $ sh -c 'cd; echo $PWD' cd: HOME not set /tmp $ HOME=/bin sh -c 'cd; echo $PWD' /bin $ HOME='' sh -c 'cd; echo $PWD' /tmp $ HOME='foo' sh -c 'cd; echo $PWD' cd: foo: No such file or directory /tmp
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi Ken, > > So an unset HOME is allowed by this function, it's an empty HOME > > which isn't. > > It strikes me as strange that there is a difference between an unset > HOME and an empty HOME in terms of behavior. I mean, yes, I can see > how the code is written, the historical precedent and how we got here, > but ... well, I'm trying to understand the justification for treating > those differently. A non-empty HOME is clear; its content should be used. An unset HOME is clear; it cannot be used. There's a ready fallback, getpwuid(3) and pw_dir so use that. What's the intent of an empty HOME? Is it set by accident when it's meant to be unset? Is it empty by accident when it's meant to be non-empty? Do they want HOME=/, HOME=$PWD, or are they expecting it to error. Any choice could be not what the user intended so exit. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
>So an unset HOME is allowed by this function, it's an empty HOME which >isn't. It strikes me as strange that there is a difference between an unset HOME and an empty HOME in terms of behavior. I mean, yes, I can see how the code is written, the historical precedent and how we got here, but ... well, I'm trying to understand the justification for treating those differently. --Ken
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi kre, Thanks for your learned input on this. As I said in another reply just now, where I listed set_mypath(), HOME being unset is fine as getpwuid() is the fallback in which case pw_dir must be non-empty. > > Alexander does point out that HOME is supposed to be valid according > > to POSIX, > > That's actually not correct, all the text quoted requires is that HOME > be set at login time - nothing at all about what can be done with it > after that. Agreed. Nearby, is: 8.1 Environment Variable Definition ... Some are placed into the environment by the implementation at the time the user logs in; all can be added or changed by the user or any ancestor of the current process. Does ‘changed’ include unsetting them? I'd expect the spec. to be precise and list ‘unset’ to clarify if they can. > Further POSIX reading will also find, in the description of ~ > expansion in the shell (XCU 2.6.1 in Issue 7 TC2 - the current version > from 2018): > > If the login name is null (that is, the tilde-prefix contains only > the tilde), the tilde-prefix is replaced by the value of the > variable HOME. If HOME is unset, the results are unspecified. ... > Setting HOME='' isn't even mentioned: in that case ~/foo expands to > /foo (which is actually what you want Agreed, it isn't mentioned. > Similarly, in XCU 4 in the description of the cd utility: > > 1. If no directory operand is given and the HOME environment > variable is empty or undefined, the default behavior is > implementation-defined and no further steps shall be taken. > > Again, HOME being unset is allowed there, this time with > implementation-defined results (which just means the implementation > must say what happens in the doc). Yes, but also allowed there is ‘empty’ which also triggers implementation-defined. Also, POSIX's chdir(3) says int chdir(const char *path); ... ERRORS The chdir() function shall fail if: ... ENOENT A component of path does not name an existing directory or path is an empty string. chdir(NULL) is invalid, obviously. But so is chdir("") so if an empty HOME is really to be treated as the root directory then code can't do h = getenv("HOME"); r = chdir(h ? h : "/"); but must instead r = chdir(h && *h ? h : "/"); I expect lots of non-nmh code doesn't do that extra check so the chdir() has no effect and the error return may not even be tested so an empty HOME results in the current directory being used rather than /. > Running with HOME unset is not even all that unusual I think it is a bit. :-) > "env -i command" results in that Agreed, my self-at(1)-ing shell scripts run at(1) under ‘env -i’ but bequeath HOME, LANG, LOGNAME, and PATH at the same time. > (and I'd even retain: relative to / if HOME is not set, which it can > be when there as been no login - like when you're in single user > mode). It would be nice for nmh to work in single-user mode as reading email might be vital in fixing problems. As long as HOME is unset and pw_dir is valid then all is well at the moment. > I'd also suggest never issuing errors because the user's environment > isn't like we think it ought to be, unless that actually prevents > something functioning as intended. The code I replaced mislead the user as to the cause of the exit. That's fixed and no HOME is okay. The possible errors are now environment variable HOME is empty password entry not found [perhaps with ‘getpwuid() failed’ and errno if it did] password entry has empty home directory This close to a release, I think we should stick with requiring HOME to be non-empty if it's set as otherwise there's too many paths to consider which the test harness probably doesn't exercise. > But I agree that a relative Path in MH should be relative to HOME Yes, I think it's too late to alter that. Though ‘path: mail’ would be a more Unixy default. :-) -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi az, > "Author: Ralph Corderoy > Date: Thu May 13 13:46:20 2021 +0100 > > sbr/path.c: add set_mypath() to factor out repeated code." I think it's worth expanding on that. commit d8ca46fabc26469be325b73a73dcc26e70681eb5 Author: Ralph Corderoy Date: Thu May 13 13:46:20 2021 +0100 sbr/path.c: add set_mypath() to factor out repeated code. set_mypath() sets the existing global ‘mypath’ which really holds the home-directory's path. As getenv(3) and getpwuid(3)'s result can be invalidated by further library calls, a value they return is mh_xstrdup()'d; not all the old bits of code replaced by a call to set_mypath() were doing this. Also add more explicit error messages if set_mypath() has trouble finding the home directory's path. And here's the current code I wrote for set_mypath() which is causing the trouble. /* set_mypath sets the global mypath to the HOME environment variable if * it is set and not empty, or else the password entry's home-directory * field if it's found and not empty. Otherwise, the program exits * with an error. The value found is copied with mh_xstrdup() as later * library calls may invalidate returned values. */ void set_mypath(void) { char *var = getenv("HOME"); if (var) { if (!*var) die("environment variable HOME is empty"); mypath = mh_xstrdup(var); return; } errno = 0; struct passwd *pw = getpwuid(getuid()); if (!pw) { if (errno) adios("", "getpwuid() failed"); /* "" prints errno! */ die("password entry not found"); } if (!*pw->pw_dir) die("password entry has empty home directory"); mypath = mh_xstrdup(pw->pw_dir); } So an unset HOME is allowed by this function, it's an empty HOME which isn't. An unset HOME falls back on getpwuid() which must dish up a non-empty pw_dir. I think that's reasonble. > a quick glance seems to indicate that only context_read in > sbr/context_read.c looks at $MH for finding a home And uip/install-mh.c in a similar manner. > but it does (well attempts) that just a few lines AFTER set_mypath has > bailed out on it... > > i think we need to fix that sequence up a little :-) It's the long-standing arrangement going back to Git's docs/historical/mh-6.8.5/sbr/m_getdefs.c which allows an empty HOME but not an empty pw_dir, though it complains that the later is the lack of a HOME ‘envariable’. :-) if (mypath == NULL) { if (mypath = getenv ("HOME")) mypath = getcpy (mypath); else if ((pw = getpwuid (getuid ())) == NULL || pw -> pw_dir == NULL || *pw -> pw_dir == 0) adios (NULLCP, "no HOME envariable"); else mypath = getcpy (pw -> pw_dir); if ((cp = mypath + strlen (mypath) - 1) > mypath && *cp == '/') *cp = 0; } It looks to me like code assumes mypath isn't NULL, e.g. exmaildir(), so not bothering to call set_mypath() if MH is set doesn't look a goer. At least not without a considerably more analysis which would probably inspire changes which would ‘disappoint’ RC-maker David. :-) -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi David, > I'd like to get Ralph's take on what we should do. Thanks. If my suggestion to Stephen of unsetting HOME works and is acceptable to him then I suggest we don't change nmh for this release. > > A further documentation issue: mh-profile(5) does not specify the > > treatment of a relative Path. I expected it to be relative to the > > directory of the profile file, but it seems to actually be relative > > to HOME. > > Right, it's relative to HOME. I'll add this clarification to the man > page: > A relative Path is relative to the user's $HOME directory. > unless there's another suggestion. As HOME needn't be set how about mh_profile(5) saying Path: Mail Locates nmh transactions in directory “Mail”. +A relative path starts from the user's home directory. This is the only mandatory profile entry. (profile, no default) I see the ENVIRONMENT section doesn't mention HOME. Perhaps, HOME If set, it is used as the user's home directory. If not set then getpwuid(3)'s pw_dir is used. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Hi Stephen, > I have investigated the failure of the xlbiff tests with nmh 1.8RC2. Thanks. > $ printf 'Path: /tmp\n' > /tmp/mh-profile-minimal > $ HOME= MH=/tmp/mh-profile-minimal /usr/bin/mh/mhparam path My suggestion for a quick fix to try is to not have HOME in the environment so getpwuid(3) is used. $ HOME= MH=$PWD/mh_profile mhparam path mhparam: environment variable HOME is empty $ $ env -u HOME MH=$PWD/mh_profile mhparam path /tmp > This is a regression. HOME is used only to set the default profile > file to "$HOME/.mh_profile". But nmh doesn't need HOME if MH is set. I think nmh *as a whole* does need HOME to be non-empty if it's set. -- Cheers, Ralph.
Re: nmh 1.8RC2, xlbiff, and $HOME
Ken Hornstein wrote: > >$ printf 'Path: /tmp\n' > /tmp/mh-profile-minimal > >$ HOME= MH=/tmp/mh-profile-minimal /usr/bin/mh/mhparam path > > Thank you for the analysis. I am wondering, though ... WHY does xlbiff > set HOME to '' for this test? To be clear, it is not xlbiff itself that does this, only an automated unit test. The test setup sets HOME to create a hermetic testing environment; the result of the test should not depend on the user running the test. My current test sets HOME to an empty string. The test could just as effectively set HOME to an non-existent directory. I know of other test suites that use this approach. This would still give me a hermetic, repeatable test, while leaving nmh more freedom to make, test, and enforce decisions about what values of HOME are allowed. < Stephen
Re: nmh 1.8RC2, xlbiff, and $HOME
Stephen wrote: > My analysis: > > This is a regression. HOME is used only to set the default profile > file to "$HOME/.mh_profile". But nmh doesn't need HOME if MH is set. Thanks for your analysis. I'd like to get Ralph's take on what we should do. > A further documentation issue: mh-profile(5) does not specify the > treatment of a relative Path. I expected it to be relative to the > directory of the profile file, but it seems to actually be relative to > HOME. Right, it's relative to HOME. I'll add this clarification to the man page: A relative Path is relative to the user's $HOME directory. unless there's another suggestion. David
Re: nmh 1.8RC2, xlbiff, and $HOME
Date:Mon, 30 Jan 2023 16:11:26 -0500 From:Ken Hornstein Message-ID: <20230130211131.8e81d1df...@pb-smtp20.pobox.com> | Alexander does point out that HOME is supposed to be | valid according to POSIX, That's actually not correct, all the text quoted requires is that HOME be set at login time - nothing at all about what can be done with it after that. Further POSIX reading will also find, in the description of ~ expansion in the shell (XCU 2.6.1 in Issue 7 TC2 - the current version from 2018): If the login name is null (that is, the tilde-prefix contains only the tilde), the tilde-prefix is replaced by the value of the variable HOME. If HOME is unset, the results are unspecified. which clearly allows the possibility that HOME not be set (which in this case makes the result of the ~ expansion unspecified, because some shells look up the password file, and use the home directory field from there, others give an error, perhaps others just use "" - but that is irrelevant to the question of whether HOME is required to be unset.) Setting HOME='' isn't even mentioned: in that case ~/foo expands to /foo (which is actually what you want, HOME=/ might (should, by a strict reading of the standard) result in //foo which is an unspecified path). Similarly, in XCU 4 in the description of the cd utility: 1. If no directory operand is given and the HOME environment variable is empty or undefined, the default behavior is implementation-defined and no further steps shall be taken. Again, HOME being unset is allowed there, this time with implementation-defined results (which just means the implementation must say what happens in the doc). Running with HOME unset is not even all that unusual, "env -i command" results in that - I do almost all builds that way, to ensure that nothing in my local environment, or home directory, affects the results, so they will be repeatable. But I agree that a relative Path in MH should be relative to HOME (and I'd even retain: relative to / if HOME is not set, which it can be when there as been no login - like when you're in single user mode). I'd also suggest never issuing errors because the user's environment isn't like we think it ought to be, unless that actually prevents something functioning as intended. kre
Re: nmh 1.8RC2, xlbiff, and $HOME
>$ printf 'Path: /tmp\n' > /tmp/mh-profile-minimal >$ HOME= MH=/tmp/mh-profile-minimal /usr/bin/mh/mhparam path Thank you for the analysis. I am wondering, though ... WHY does xlbiff set HOME to '' for this test? (I am neutral on whether or not this is technically a regression; I can see it both ways. Alexander does point out that HOME is supposed to be valid according to POSIX, but I am not 100% that means it can't be a zero-length string). --Ken
Re: nmh 1.8RC2, xlbiff, and $HOME
On Mon, 30 Jan 2023 10:47:05 -0800, Stephen Gildea writes: >I have investigated the failure of the xlbiff tests with nmh 1.8RC2. >(This is https://bugs.debian.org/1029752) stephen, thanks for that. >In one of the tests, xlbiff sets environment variable HOME to an empty >string and MH to a file containing a custom profile with an absolute Path. >With nmh 1.7.1, this environment works. > >With nmh 1.8RC2, this environment fails with the error message >"environment variable HOME is empty". *lightbulb* that error message comes from set_mypath in sbr/path.c, and the changelog tells me that "Author: Ralph Corderoy Date: Thu May 13 13:46:20 2021 +0100 sbr/path.c: add set_mypath() to factor out repeated code." started to consolidate the various places that figure out a home for mh. however, set_mypath spot doesn't bother with $MH. a quick glance seems to indicate that only context_read in sbr/context_read.c looks at $MH for finding a home, but it does (well attempts) that just a few lines AFTER set_mypath has bailed out on it... i think we need to fix that sequence up a little :-) [handling of relative paths in the profile] >Whatever your decision, the choice should be documented. If you decide >to keep Path relative to HOME, then HOME should be required to be >non-empty if (and only if!) Path is relative. (nmh 1.7.1 used "/", >which seems wrong.) with my nitpicker's hat on: POSIX/SUS says HOME can be depended on. "HOME The system shall initialize this variable at the time of login to be a pathname of the user's home directory. See ." personally i'd say we should rely on posix and insist on not being $HOMEless. for me that would imply relatives should be relative to that, not the profile location. regards az -- Alexander Zangerl + GPG Key 2FCCF66BB963BD5F + https://snafu.priv.at/ `bastard operators from hell' anagrams to `shatterproof armored balls'. -- Cliff Miller signature.asc Description: Digital Signature
Re: nmh 1.8RC2, xlbiff, and $HOME
I also use xlbiff and would like it to continue working without needing to fuss with it. The next best option for me is that I might be required to set some environment variables in a sane way that doesn't cascade into the operation of other programs. Thanks for uncovering this release client change/issue! On Mon, Jan 30, 2023 at 10:47 AM Stephen Gildea wrote: > I have investigated the failure of the xlbiff tests with nmh 1.8RC2. > (This is https://bugs.debian.org/1029752) > > In one of the tests, xlbiff sets environment variable HOME to an empty > string and MH to a file containing a custom profile with an absolute Path. > With nmh 1.7.1, this environment works. > > With nmh 1.8RC2, this environment fails with the error message > "environment variable HOME is empty". > > You can see the change in behavior as follows: > > $ printf 'Path: /tmp\n' > /tmp/mh-profile-minimal > $ HOME= MH=/tmp/mh-profile-minimal /usr/bin/mh/mhparam path > > > nmh 1.7.1: > /tmp > > nmh 1.8RC2: > mhparam: environment variable HOME is empty > > > My analysis: > > This is a regression. HOME is used only to set the default profile > file to "$HOME/.mh_profile". But nmh doesn't need HOME if MH is set. > > A further documentation issue: mh-profile(5) does not specify the > treatment of a relative Path. I expected it to be relative to the > directory of the profile file, but it seems to actually be relative to > HOME. > > Whatever your decision, the choice should be documented. If you decide to > keep Path relative to HOME, then HOME should be required to be non-empty if > (and only if!) Path is relative. (nmh 1.7.1 used "/", which seems wrong.) > > < Stephen > >
nmh 1.8RC2, xlbiff, and $HOME
I have investigated the failure of the xlbiff tests with nmh 1.8RC2. (This is https://bugs.debian.org/1029752) In one of the tests, xlbiff sets environment variable HOME to an empty string and MH to a file containing a custom profile with an absolute Path. With nmh 1.7.1, this environment works. With nmh 1.8RC2, this environment fails with the error message "environment variable HOME is empty". You can see the change in behavior as follows: $ printf 'Path: /tmp\n' > /tmp/mh-profile-minimal $ HOME= MH=/tmp/mh-profile-minimal /usr/bin/mh/mhparam path nmh 1.7.1: /tmp nmh 1.8RC2: mhparam: environment variable HOME is empty My analysis: This is a regression. HOME is used only to set the default profile file to "$HOME/.mh_profile". But nmh doesn't need HOME if MH is set. A further documentation issue: mh-profile(5) does not specify the treatment of a relative Path. I expected it to be relative to the directory of the profile file, but it seems to actually be relative to HOME. Whatever your decision, the choice should be documented. If you decide to keep Path relative to HOME, then HOME should be required to be non-empty if (and only if!) Path is relative. (nmh 1.7.1 used "/", which seems wrong.) < Stephen