Re: Problem with mutt version.sh script

2018-04-26 Thread Vincent Lefevre 
On 2018-04-23 14:25:29 -0500, Derek Martin wrote:
> [Then again, I made the same arguments about binutils removing the
> -[number] option to head and tail, and the maintainers went ahead and
> did it anyway. Of course, they finally realized their mistake, and PUT
> IT BACK IN.]

This is different: there was no good reason to remove -[number]
(it's rather clean, contrary to test's -o). And it is also very
useful in interactive use as this is a bit shorter (thus faster
to type) than -n [number].

> In general, the only way the compount -o  wouldn't work is if you
> write some syntax where the shell can't disambiguate the binary
> operator from the unary operator.  It's actually somewhat hard to do
> this unintentionally,

No, not hard: it suffices that the pathname be arbitrary, e.g. an
argument provided by the user (who may not be the author and owner of
this script) or some existing pathname (e.g. created by a malicious
user).

> and easy to avoid...  The primary way this would
> happen is if you didn't quote a shell variable which was empty, though
> that will often also cause other related syntax errors.  There are
> probably other ways.  All of them can be avoided if you use parens to
> group your expressions (but you have to be careful to quote the parens
> properly).

But almost no-ones use parens.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Problem with mutt version.sh script

2018-04-23 Thread Derek Martin 
On Mon, Apr 23, 2018 at 01:37:31PM +0200, Vincent Lefevre wrote:
> On 2018-04-20 17:01:50 -0500, Derek Martin wrote:
> > On Thu, Apr 19, 2018 at 08:24:43PM +0200, Vincent Lefevre wrote:
> > > On 2018-04-17 14:28:17 -0500, Derek Martin wrote:
> > > >
> > > >   [ -f $file -o -d $file ]
> > > 
> > > AFAIK, -o is not portable and has been marked obsolescent.
> > > You should write:
[...]
> For POSIX, this is an XSI extension. Thus not all POSIX shells might
> support it. And in practice, its use in the general case is error
> prone and the behavior depends on the shell. So, the best thing is
> probably to have it removed in the future.

We have multiple equivalent solutions that should work everywhere, so
again I'm continuing here purely as an academic exercise...

The short version is things that are deprecated rarely get removed in
practice, because Legacy.  The longer version:

There's what the standard says, and there's what is implemented in
reality.  Case in point: RFC 2047 headers.  The spec specifically
forbids them in certain headers, but due to a certain badly-behaved
corporate entity forcing our hand to maintain compatibility (and in
this case, not unreasonably I might add), the world has pretty
universally implemented them in those headers anyway.

This case is rather like that... the compound comparison was
implemented in the original Bourne shell, and any shell that wanted to
maintain compatibility (i.e. all of them, if they were based on Bourne
syntax) had to implement that syntax to be compatible.  I understated
this earlier, but I personally used to use this syntax frequently in
my shell scripts, which I have written on and/or ported to Ultrix,
OSF/1 (or Dec Unix), HP-Ux 9, HP-UX 10+, SunOS, Solaris, Bash (on many
platforms), and probably a few I'm forgetting.  It's prohibitive to
determine exhaustively its availability, but if there were a
Bourne-like shell where this syntax was not available, I would be
extremely, extremely suprised.  Removing it has the potential to break
thousands if not millions of shell scripts.  And after all, who pays
such close attention to the standard specs?  To be perfectly honest
Vincent, in my many years working in tech, you are quite literally the
only person I've come across that does...

[Then again, I made the same arguments about binutils removing the
-[number] option to head and tail, and the maintainers went ahead and
did it anyway. Of course, they finally realized their mistake, and PUT
IT BACK IN.]

In general, the only way the compount -o  wouldn't work is if you
write some syntax where the shell can't disambiguate the binary
operator from the unary operator.  It's actually somewhat hard to do
this unintentionally, and easy to avoid...  The primary way this would
happen is if you didn't quote a shell variable which was empty, though
that will often also cause other related syntax errors.  There are
probably other ways.  All of them can be avoided if you use parens to
group your expressions (but you have to be careful to quote the parens
properly).  But the case where we'd use this clearly doesn't suffer
from any of the possible pitfalls, provided any variables are quoted
properly.

On the other hand, as I've already hinted, the only known case of -e
not working is on an OS which is itself woefully obsolete, so I'd be
just as happy if Mutt said "tough noogies, or patch it yourself."

=8^)

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpnEHp9GOmUl.pgp
Description: PGP signature

Re: Problem with mutt version.sh script

2018-04-23 Thread Vincent Lefevre 
On 2018-04-20 17:01:50 -0500, Derek Martin wrote:
> On Thu, Apr 19, 2018 at 08:24:43PM +0200, Vincent Lefevre wrote:
> > On 2018-04-17 14:28:17 -0500, Derek Martin wrote:
> > > One wonders what systems these are, and whether this is even worth
> > > considering.  If it is, I believe Bourne shell allows compound tests
> > > like the following: 
> > > 
> > >   [ -f $file -o -d $file ]
> > 
> > AFAIK, -o is not portable and has been marked obsolescent.
> > You should write:
> > 
> >   [ -f "$file" ] || [ -d "$file" ]
> 
> Sure, I have no issue with writing it that way... it's probably
> slighly less efficient, but whether it is or isn't is not interesting
> here.
> 
> I have doubts about the "not portable" comment though...  AFAICT it's
> currently supported by Bourne, Korn, Bash, and all POSIX-compliant
> shells.

For POSIX, this is an XSI extension. Thus not all POSIX shells might
support it. And in practice, its use in the general case is error
prone and the behavior depends on the shell. So, the best thing is
probably to have it removed in the future.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Problem with mutt version.sh script

2018-04-20 Thread Derek Martin 
On Thu, Apr 19, 2018 at 08:24:43PM +0200, Vincent Lefevre wrote:
> On 2018-04-17 14:28:17 -0500, Derek Martin wrote:
> > One wonders what systems these are, and whether this is even worth
> > considering.  If it is, I believe Bourne shell allows compound tests
> > like the following: 
> > 
> >   [ -f $file -o -d $file ]
> 
> AFAIK, -o is not portable and has been marked obsolescent.
> You should write:
> 
>   [ -f "$file" ] || [ -d "$file" ]

Sure, I have no issue with writing it that way... it's probably
slighly less efficient, but whether it is or isn't is not interesting
here.

I have doubts about the "not portable" comment though...  AFAICT it's
currently supported by Bourne, Korn, Bash, and all POSIX-compliant
shells.  I'm not sure where it wouldn't port to, other than to shells
which don't have Bourne-like syntax in the first place.  Arguing that
the syntax is obsolete, when the point is to support an entire shell
that is obsolete, seems redundant.  But I think since both versions of
the syntax should be well supported it's purely academic.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpeQw2u4cdBj.pgp
Description: PGP signature

Re: Problem with mutt version.sh script

2018-04-19 Thread Vincent Lefevre 
On 2018-04-17 14:28:17 -0500, Derek Martin wrote:
> One wonders what systems these are, and whether this is even worth
> considering.  If it is, I believe Bourne shell allows compound tests
> like the following: 
> 
>   [ -f $file -o -d $file ]

AFAIK, -o is not portable and has been marked obsolescent.
You should write:

  [ -f "$file" ] || [ -d "$file" ]

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Subject: Re: Problem with mutt version.sh script

2018-04-18 Thread Derek Martin 
On Wed, Apr 18, 2018 at 07:43:34AM -0500, Paul Keusemann wrote:
> Trimming digest, replying inline below...
> 
> On Wed, Apr 18, 2018 at 7:00 AM,  wrote:
> 
> >
> >
> > Message: 4
> > Date: Tue, 17 Apr 2018 14:28:17 -0500
> > From: Derek Martin 
> > To: mutt-dev@mutt.org
> > Subject: Re: Problem with mutt version.sh script
> > Message-ID: <20180417192817.ge13...@bladeshadow.org>
> > Content-Type: text/plain; charset="utf-8"
> >
> > On Mon, Apr 16, 2018 at 09:26:51AM -0700, Kevin J. McCarthy wrote:
> > > On Mon, Apr 16, 2018 at 10:18:57AM -0400, Aaron Schrab wrote:
> > > > At 15:41 +0200 16 Apr 2018, Gero Treuner 
> > wrote:
> > > > > On Mon, Apr 16, 2018 at 07:53:56AM -0500, Paul Keusemann wrote:
> > > > > > { [ -e ".git" ] && command -v git >/dev/null 2>&1; } \
> > > > > > || exec cat VERSION
> > > > > >
> > > > > > does not work when run in a bourne shell.? The -e option is not
> > supported in
> > [...]
> > > Whoops, thank you Paul for pointing this out.
> >
> > One wonders what systems these are, and whether this is even worth
> > considering.  If it is, I believe Bourne shell allows compound tests
> > like the following:
> >
> >   [ -f $file -o -d $file ]
> >
> 
> This is on Solaris 10.  According to the man page for test, the -o 
> option is used to check whether or not  is on.

Yes, but keep reading.  

[For the peanut gallery, note that the relevant man page *seems to be*
the one for /usr/bin/test on Solaris 10.  That's a bit dubious as most
shells implement test as a built-in (in addition to having the on-disk
implementation) which may or may not behave the same way.  It does
seem to mention shell-specific syntax though, in at least a few cases.
All that said, looking at that man page...]

It also has 

  condition1 -o condition2

Which is what I was referring to.  I'm quite positive I've used this
syntax while doing (genuine) Bourne shell scripting at some point,
most likely on Ultrix circa 1994, but since it was so long ago I can't
be exactly sure when or which exact shell and which exact features
were supported.  Nevertheless, the Solaris 10 man page seems to
confirm that it should work.  I will add a caveat that it may not work
as expected if the parser can't disambiguate the two variants of -o
usage, which may depend on your exact syntax (and how buggy your
Bourne shell is).

Solaris 10 is 16 years old. =8^(  It is sadly not EOL until 2021, but
it's already pretty darn long obsolete.  Case in point.

> > [Although, FWIW, I could have sworn Bourne shell supported -e as
> > well...]
> 
> According to the man page, not available in sh.

Indeed.  But I'm almost certain I've used it with some Bourne shell or
other... my best guess would be HP-UX 9, but it's possible I'm
mis-remembering and it was actually the POSIX shell on HP-UX 10 or
later, or OSF-1, or some other OS entirely.  This would have been in
the time frame of roughly 1994-1998.  The POSIX shell became standard
so long ago that frankly I think Sun should be embarrassed their
default shell isn't POSIX-compatible.

> This is on Solaris 10, x86 and SPARC.  I don't currently have a Solaris 11
> machine running but hope to soon.  Once I get it up and running I can test
> there as well.

All of the suggested options, including the original that caused this
thread, should work correctly on Solaris 11, though the point is moot
since Kevin made the change.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpios8drDewa.pgp
Description: PGP signature

Subject: Re: Problem with mutt version.sh script

2018-04-18 Thread Paul Keusemann 
Trimming digest, replying inline below...

On Wed, Apr 18, 2018 at 7:00 AM,  wrote:

>
>
> Message: 4
> Date: Tue, 17 Apr 2018 14:28:17 -0500
> From: Derek Martin 
> To: mutt-dev@mutt.org
> Subject: Re: Problem with mutt version.sh script
> Message-ID: <20180417192817.ge13...@bladeshadow.org>
> Content-Type: text/plain; charset="utf-8"
>
> On Mon, Apr 16, 2018 at 09:26:51AM -0700, Kevin J. McCarthy wrote:
> > On Mon, Apr 16, 2018 at 10:18:57AM -0400, Aaron Schrab wrote:
> > > At 15:41 +0200 16 Apr 2018, Gero Treuner 
> wrote:
> > > > On Mon, Apr 16, 2018 at 07:53:56AM -0500, Paul Keusemann wrote:
> > > > > { [ -e ".git" ] && command -v git >/dev/null 2>&1; } \
> > > > > || exec cat VERSION
> > > > >
> > > > > does not work when run in a bourne shell.? The -e option is not
> supported in
> [...]
> > Whoops, thank you Paul for pointing this out.
>
> One wonders what systems these are, and whether this is even worth
> considering.  If it is, I believe Bourne shell allows compound tests
> like the following:
>
>   [ -f $file -o -d $file ]
>

This is on Solaris 10.  According to the man page for test, the -o 
option is used to check whether or not  is on.



>
> Message: 5
> Date: Tue, 17 Apr 2018 14:32:20 -0500
> From: Derek Martin 
> To: mutt-dev@mutt.org
> Subject: Re: Problem with mutt version.sh script
> Message-ID: <20180417193220.gf13...@bladeshadow.org>
> Content-Type: text/plain; charset="utf-8"
>
> On Tue, Apr 17, 2018 at 02:28:17PM -0500, Derek Martin wrote:
> > I believe Bourne shell allows compound tests like the following:
> >
> >   [ -f $file -o -d $file ]
>
> [Although, FWIW, I could have sworn Bourne shell supported -e as
> well...]
>

According to the man page, not available in sh.



>
>
> Message: 6
> Date: Tue, 17 Apr 2018 18:01:09 -0700
> From: "Kevin J. McCarthy" 
> To: mutt-dev@mutt.org
> Subject: Re: Problem with mutt version.sh script
> Message-ID: <20180418010109.gd9...@afu.lan>
> Content-Type: text/plain; charset="us-ascii"
>
> On Tue, Apr 17, 2018 at 02:28:17PM -0500, Derek Martin wrote:
> > Wasn't there some recent-ish notion that Mutt would now require
> > vaguely modern (i.e. C99-compliant) systems to compile?  If so,
> > doesn't that implicitly include having a /bin/sh that is POSIX?
>
> Yes, I (we) have been trying to move in that direction.  So perhaps it's
> a step in the wrong direction to support non-posix shells, when Mutt
> already requires a new compiler, SSL libraries, etc.
>
> I think because the change is so tiny, I don't mind bending a bit.  I've
> already pushed a change to [-r], but if there turn out to be issues I'll
> go with your suggestion, Derek.
>
> Paul, for everyone's curiosity, is this the only issue you've hit with
> compilation?  Also, what system are you running?
>

This is on Solaris 10, x86 and SPARC.  I don't currently have a Solaris 11
machine running but hope to soon.  Once I get it up and running I can test
there as well.


>
>
-- 
Paul Keusemann   pkeu...@gmail.com
4266 Joppa Court (952) 894-7805
Savage, MN  55378

Re: Problem with mutt version.sh script

2018-04-17 Thread Kevin J. McCarthy 
On Tue, Apr 17, 2018 at 02:28:17PM -0500, Derek Martin wrote:
> Wasn't there some recent-ish notion that Mutt would now require
> vaguely modern (i.e. C99-compliant) systems to compile?  If so,
> doesn't that implicitly include having a /bin/sh that is POSIX?

Yes, I (we) have been trying to move in that direction.  So perhaps it's
a step in the wrong direction to support non-posix shells, when Mutt
already requires a new compiler, SSL libraries, etc.

I think because the change is so tiny, I don't mind bending a bit.  I've
already pushed a change to [-r], but if there turn out to be issues I'll
go with your suggestion, Derek.

Paul, for everyone's curiosity, is this the only issue you've hit with
compilation?  Also, what system are you running?

-- 
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature

Re: Problem with mutt version.sh script

2018-04-17 Thread Derek Martin 
On Tue, Apr 17, 2018 at 02:28:17PM -0500, Derek Martin wrote:
> I believe Bourne shell allows compound tests like the following: 
> 
>   [ -f $file -o -d $file ]

[Although, FWIW, I could have sworn Bourne shell supported -e as
well...]

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpBZf28YbL4s.pgp
Description: PGP signature

Re: Problem with mutt version.sh script

2018-04-17 Thread Derek Martin 
On Mon, Apr 16, 2018 at 09:26:51AM -0700, Kevin J. McCarthy wrote:
> On Mon, Apr 16, 2018 at 10:18:57AM -0400, Aaron Schrab wrote:
> > At 15:41 +0200 16 Apr 2018, Gero Treuner  wrote:
> > > On Mon, Apr 16, 2018 at 07:53:56AM -0500, Paul Keusemann wrote:
> > > > { [ -e ".git" ] && command -v git >/dev/null 2>&1; } \
> > > > || exec cat VERSION
> > > > 
> > > > does not work when run in a bourne shell.  The -e option is not 
> > > > supported in
[...]
> Whoops, thank you Paul for pointing this out.

One wonders what systems these are, and whether this is even worth
considering.  If it is, I believe Bourne shell allows compound tests
like the following: 

  [ -f $file -o -d $file ]

which I believe would cover it, though I don't have access to a
machine that currently has an original Bourne shell to test.  It is
becoming rather difficult to find on-line references to syntax for the
(now obsolete) original Bourne shell.  Most such refereces I can find
actually refer to bash, ksh, or other POSIX shell variants.

Wasn't there some recent-ish notion that Mutt would now require
vaguely modern (i.e. C99-compliant) systems to compile?  If so,
doesn't that implicitly include having a /bin/sh that is POSIX?

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpgIBpWSusot.pgp
Description: PGP signature

Re: Problem with mutt version.sh script

2018-04-17 Thread Aaron Schrab 

At 15:27 +0200 17 Apr 2018, Vincent Lefevre  wrote:

I suppose that -r should work since .git must be readable to be
useful.


That's not actually true. When it's a directory, a large number of 
operations can be done without it being readable (in my testing I 
haven't come across anything that fails).  Since git is usually dealing 
with known entries within that directory it doesn't need to list them.  
So, only execute (search) permission is necessary for many things.


But, I don't think that this is a likely occurrence. It's probably still 
a better idea to work around shells which don't support the `-e` test 
even if that violates POSIX.

Re: Problem with mutt version.sh script

2018-04-17 Thread Vincent Lefevre 
On 2018-04-16 07:53:56 -0500, Paul Keusemann wrote:
> I have just built mutt-1.9.5 on several platforms and ran into a problem
> with the version.sh script on systems where /bin/sh is not bash.

I suppose you meant... where /bin/sh is not a POSIX shell.

On 2018-04-16 09:26:51 -0700, Kevin J. McCarthy wrote:
> Originally, the test was '-d', but as Aaron notes, .git can be a file
> too.  I thought having dash as /bin/sh would catch these issues but
> somehow the '-e' problem slipped past us.

It would be a bug if -e were not supported under dash since -e is
specified by POSIX.

> Aaron, your suggestion of '-r' looks like a good idea.  Does anyone know
> of a problem with that?  Otherwise I'll swap that out in version.sh and
> mkreldate.sh.

I suppose that -r should work since .git must be readable to be
useful.

-- 
Vincent Lefèvre  - Web: 
100% accessible validated (X)HTML - Blog: 
Work: CR INRIA - computer arithmetic / AriC project (LIP, ENS-Lyon)

Re: Problem with mutt version.sh script

2018-04-16 Thread Kevin J. McCarthy 
On Mon, Apr 16, 2018 at 10:18:57AM -0400, Aaron Schrab wrote:
> At 15:41 +0200 16 Apr 2018, Gero Treuner  wrote:
> > On Mon, Apr 16, 2018 at 07:53:56AM -0500, Paul Keusemann wrote:
> > > { [ -e ".git" ] && command -v git >/dev/null 2>&1; } \
> > > || exec cat VERSION
> > > 
> > > does not work when run in a bourne shell.  The -e option is not supported 
> > > in
> 
> > I'd prefer using an option which is supported by Bourne shell. What
> > about "-d"?
> 
> The `.git` entry won't always be a directory. There are a few ways that it
> could be a file instead (`git clone --separate-git-dir` and `git worktree
> add` are a couple ways).
> 
> I believe that `-r` is supported by the Bourne shell as well, and should
> handle `.git` being either a file or a directory.

Whoops, thank you Paul for pointing this out.

Originally, the test was '-d', but as Aaron notes, .git can be a file
too.  I thought having dash as /bin/sh would catch these issues but
somehow the '-e' problem slipped past us.

Aaron, your suggestion of '-r' looks like a good idea.  Does anyone know
of a problem with that?  Otherwise I'll swap that out in version.sh and
mkreldate.sh.

-- 
Kevin J. McCarthy
GPG Fingerprint: 8975 A9B3 3AA3 7910 385C  5308 ADEF 7684 8031 6BDA


signature.asc
Description: PGP signature

Re: Problem with mutt version.sh script

2018-04-16 Thread Aaron Schrab 

At 15:41 +0200 16 Apr 2018, Gero Treuner  wrote:

On Mon, Apr 16, 2018 at 07:53:56AM -0500, Paul Keusemann wrote:

{ [ -e ".git" ] && command -v git >/dev/null 2>&1; } \
|| exec cat VERSION

does not work when run in a bourne shell.  The -e option is not supported in



I'd prefer using an option which is supported by Bourne shell. What
about "-d"?


The `.git` entry won't always be a directory. There are a few ways that 
it could be a file instead (`git clone --separate-git-dir` and `git 
worktree add` are a couple ways).


I believe that `-r` is supported by the Bourne shell as well, and should 
handle `.git` being either a file or a directory.

Re: Problem with mutt version.sh script

2018-04-16 Thread Gero Treuner 
Hi,

On Mon, Apr 16, 2018 at 07:53:56AM -0500, Paul Keusemann wrote:
> I have just built mutt-1.9.5 on several platforms and ran into a problem
> with the version.sh script on systems where /bin/sh is not bash.
> 
> This:
> 
> { [ -e ".git" ] && command -v git >/dev/null 2>&1; } \
> || exec cat VERSION
> 
> does not work when run in a bourne shell.  The -e option is not supported in
> the version of test built in to the bourne shell. It does work when run in
> bash or a Korn shell.
> 
> This:
> 
> { /bin/test -e ".git" && command -v git >/dev/null 2>&1; } \
> || exec cat VERSION
> 
> does work in a bourne shell.

I'd prefer using an option which is supported by Bourne shell. What
about "-d"?

That should be possible according to the description (with very few test
options) here:
https://en.wikibooks.org/wiki/Bourne_Shell_Scripting/Appendix_C:_Quick_Reference


Gero