[1003.1(2016)/Issue7+TC2 0001267]: sh HOME should not be UP shaded
A NOTE has been added to this issue. == http://www.austingroupbugs.net/view.php?id=1267 == Reported By:dennisw Assigned To: == Project:1003.1(2016)/Issue7+TC2 Issue ID: 1267 Category: Shell and Utilities Type: Error Severity: Comment Priority: normal Status: New Name: Dennis Wölfing Organization: User Reference: Section:sh Page Number:3228 Line Number:108396-108397 Interp Status: --- Final Accepted Text: == Date Submitted: 2019-07-03 15:24 UTC Last Modified: 2019-07-04 14:15 UTC == Summary:sh HOME should not be UP shaded == -- (0004472) shware_systems (reporter) - 2019-07-04 14:15 http://www.austingroupbugs.net/view.php?id=1267#c4472 -- Perhaps, but I think my analysis has some validity too, in that its presence is optional, so if the shading is removed it may desirable to add text accenting that aspect, e.g. Change last sentence to: The contents of HOME, when it is set and is a non-null value, are used in tilde expansion as described in [xref to XCU 2.6.1]. Issue History Date ModifiedUsername FieldChange == 2019-07-03 15:24 denniswNew Issue 2019-07-03 15:24 denniswName => Dennis Wölfing 2019-07-03 15:24 denniswSection => sh 2019-07-03 15:24 denniswPage Number => 3228 2019-07-03 15:24 denniswLine Number => 108396-108397 2019-07-03 16:22 shware_systems Note Added: 0004470 2019-07-04 08:10 geoffclare Note Added: 0004471 2019-07-04 14:15 shware_systems Note Added: 0004472 ==
Re: [1003.1(2016)/Issue7+TC2 0001234]: in most shells, backslash doesn't have two meaning wrt pattern matching
On 04/07/2019 09:09, Geoff Clare wrote: Harald van Dijk wrote, on 03 Jul 2019: No, it's a context where shell-quoting backslash *doesn't* work. Therefore the backslash should act as an escape character just like in find, pax, fnmatch() and glob(). It's not about shell quoting the backslash, it's about whether shell quoting can be used on the asterisk. The quoted sentence says that if shell quoting cannot be used on the asterisk, then it can be escaped with a backslash instead. But shell quoting can be used on the asterisk: just try a='*' ls -ld "$a" This does work for listing only a file literally named '*'. That's because of the double quotes around $a. Pathname expansion is not done inside double quotes, so there is no pattern here, just a string that contains a '*'. Is that relevant? It still shows that shell quoting can force the '*' to be treated literally, does it not? In this case, whether pathname expansion is performed does not affect the result. In the general case, words may be partly quoted, and the quoted parts of such words are definitely supposed to be used in pathname expansion, so let's go with a different example: a='*' ls -ld $a* #1 ls -ld "$a"* #2 Here, it is clear that pathname expansion is performed. #1 lists all non-hidden files, #2 lists all files starting with '*'. Since shell quoting works, under my understanding of your proposed wording, that means backslash cannot be used to escape that '*', and a='\*' ls -ld $a* must list all files starting with '\' since backslash does not function as an escape character here. It would be bizarre if the handling of '\' in $a is different from the handling of '\' in $a*. Cheers, Harald van Dijk
Re: [1003.1(2016)/Issue7+TC2 0001234]: in most shells, backslash doesn't have two meaning wrt pattern matching
Geoff Clare wrote, on 04 Jul 2019: > > Harald van Dijk wrote, on 03 Jul 2019: > > > > >No, it's a context where shell-quoting backslash *doesn't* work. Therefore > > >the backslash should act as an escape character just like in find, pax, > > >fnmatch() and glob(). > > > > It's not about shell quoting the backslash, it's about whether shell quoting > > can be used on the asterisk. The quoted sentence says that if shell quoting > > cannot be used on the asterisk, then it can be escaped with a backslash > > instead. But shell quoting can be used on the asterisk: just try > > > > a='*' > > ls -ld "$a" > > > > This does work for listing only a file literally named '*'. > > That's because of the double quotes around $a. Pathname expansion > is not done inside double quotes, so there is no pattern here, just > a string that contains a '*'. Given that the side-discussion that arose from this may well lead to rewording that does mean double quotes "affect" indirect patterns, I think it would be prudent to reword the new condition in 2.13.1. I'm thinking make it specific to backslash: In a pattern, or part of one, where a shell-quoting can be used ... In a pattern, or part of one, where a shell-quoting cannot be used (such as ... -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: Effect of double quotes on pathname expansion
Geoff Clare wrote, on 04 Jul 2019: > > I started thinking it would be better to handle this in 2.13.1, but > then realised it would be difficult to specify there given that 2.6.2 > says "Enclosing the full parameter expansion string in double-quotes > shall not cause the following four varieties of pattern characters to > be quoted, whereas quoting characters within the braces shall have this > effect." > > So I think we need to add something to that part of 2.6.2, and to > 2.9.4.3. On second thoughts, it would probably be best to handle it in 2.2.3 and get rid of all the text in other places that talks about how double quotes affect expansions done inside them. Something like: When double-quotes are used to quote a parameter expansion, command substitution or arithmetic expansion, the literal value of all characters within the result of the expansion shall be preserved. I think this works okay with the text from 2.6.2 I quoted above because that text says "shall not cause ... to be quoted". I.e. since the pattern is not "quoted", this new text does not apply to it. -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: Effect of double quotes on pathname expansion
Stephane Chazelas wrote, on 04 Jul 2019:
>
> 2019-07-04 10:34:37 +0100, Stephane Chazelas:
> [...]
> > See 2.6.2 parameter expansion:
> >
> > } If a parameter expansion occurs inside double-quotes:
> > }
> > } • Pathname expansion shall not be performed on the results of the
> > expansion.
> > }
> > } • Field splitting shall not be performed on the results of the
> > expansion.
>
> That wording is a bit wrong. I think the intention was that it
> removes the special meaning of wildcard characters, it shouldn't
> by itself affect whether pathname expansion is done or not (like
> in ls -d -- "$var"*).
I agree.
> Same applies for command substitution.
Yes.
> There's no matching text to say that word splitting is not
> applied if $((...)) is in double quotes, but IIRC, it's
> clarified elsewhere that word splitting is not performed on
> quoted expansions.
Yes, 2.6.5 covers that. (Which means mentioning it in 2.6.2 and 2.6.3
is redundant.)
> > I'm sure there's something similar for ${a#"$b"} or case $a in ("$b")
> [...]
>
> Well, I'm not sure about ${a#"$b"} now. There's this text I
> don't really understand as well:
I just mentioned this as well (our emails crossed).
> > Enclosing the full parameter expansion string in double-quotes
> > shall not cause the following four varieties of pattern
> > characters to be quoted, whereas quoting characters within the
> > braces shall have this effect. In each variety, if word is
> > omitted, the empty pattern shall be used.
>
> What "following four varieties of pattern characters"?
I think that relates to "The following four varieties of parameter
expansion" at the beginning of the paragraph, but it's certainly oddly
worded and could do with improving.
> (follows a number of examples showing bad coding practice
> (missing quotes, usage of echo for arbitrary data...))
I think these are ones that you previously touched upon (where you said
that in the past I had responded that examples can assume IFS has the
default value). If you think it is worth the bother of updating them,
then by all means submit a bug with the necessary detailed wording changes.
I don't object to changing them, I just don't think it is worth the
effort of working out the wording changes and submitting a bug.
--
Geoff Clare
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: Effect of double quotes on pathname expansion
Geoff Clare wrote: > The equivalent for command substitution is in 2.6.3 on line 75206: > > If a command substitution occurs inside double-quotes, field > splitting and pathname expansion shall not be performed on the > results of the substitution. That "does not happen" does not mean that is has been switched off, but rather that there is quoting in the intermediate result and that supresses the effect. Jörg -- EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'
Re: Effect of double quotes on pathname expansion
Stephane Chazelas wrote, on 04 Jul 2019:
>
> 2019-07-04 09:26:50 +0100, Geoff Clare:
> [...]
> > > > a='*'
> > > > ls -ld "$a"
> > > >
> > > > This does work for listing only a file literally named '*'.
> > >
> > > That's because of the double quotes around $a. Pathname expansion
> > > is not done inside double quotes, so there is no pattern here, just
> > > a string that contains a '*'.
> >
> > Having said that, I now can't find the place in the standard that
> > covers this.
> >
> > Am I just not finding it, or is it genuinely missing?
> [...]
>
> See 2.6.2 parameter expansion:
>
> } If a parameter expansion occurs inside double-quotes:
> }
> } • Pathname expansion shall not be performed on the results of the
> expansion.
> }
> } • Field splitting shall not be performed on the results of the expansion.
Thanks!
It didn't occur to me that it might be down in the descriptions of
individual expansions.
The equivalent for command substitution is in 2.6.3 on line 75206:
If a command substitution occurs inside double-quotes, field
splitting and pathname expansion shall not be performed on the
results of the substitution.
> I'm sure there's something similar for ${a#"$b"} or case $a in ("$b")
Not that I can see.
I started thinking it would be better to handle this in 2.13.1, but
then realised it would be difficult to specify there given that 2.6.2
says "Enclosing the full parameter expansion string in double-quotes
shall not cause the following four varieties of pattern characters to
be quoted, whereas quoting characters within the braces shall have this
effect."
So I think we need to add something to that part of 2.6.2, and to
2.9.4.3.
--
Geoff Clare
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
Re: Effect of double quotes on pathname expansion
2019-07-04 10:34:37 +0100, Stephane Chazelas:
[...]
> See 2.6.2 parameter expansion:
>
> } If a parameter expansion occurs inside double-quotes:
> }
> } • Pathname expansion shall not be performed on the results of the
> expansion.
> }
> } • Field splitting shall not be performed on the results of the expansion.
That wording is a bit wrong. I think the intention was that it
removes the special meaning of wildcard characters, it shouldn't
by itself affect whether pathname expansion is done or not (like
in ls -d -- "$var"*).
Same applies for command substitution.
There's no matching text to say that word splitting is not
applied if $((...)) is in double quotes, but IIRC, it's
clarified elsewhere that word splitting is not performed on
quoted expansions.
> I'm sure there's something similar for ${a#"$b"} or case $a in ("$b")
[...]
Well, I'm not sure about ${a#"$b"} now. There's this text I
don't really understand as well:
> Enclosing the full parameter expansion string in double-quotes
> shall not cause the following four varieties of pattern
> characters to be quoted, whereas quoting characters within the
> braces shall have this effect. In each variety, if word is
> omitted, the empty pattern shall be used.
What "following four varieties of pattern characters"?
(follows a number of examples showing bad coding practice
(missing quotes, usage of echo for arbitrary data...))
I can't find much about case $a in ($b) either.
--
Stephane
Re: Effect of double quotes on pathname expansion
2019-07-04 09:26:50 +0100, Geoff Clare:
[...]
> > > a='*'
> > > ls -ld "$a"
> > >
> > > This does work for listing only a file literally named '*'.
> >
> > That's because of the double quotes around $a. Pathname expansion
> > is not done inside double quotes, so there is no pattern here, just
> > a string that contains a '*'.
>
> Having said that, I now can't find the place in the standard that
> covers this.
>
> Am I just not finding it, or is it genuinely missing?
[...]
See 2.6.2 parameter expansion:
} If a parameter expansion occurs inside double-quotes:
}
} • Pathname expansion shall not be performed on the results of the expansion.
}
} • Field splitting shall not be performed on the results of the expansion.
I'm sure there's something similar for ${a#"$b"} or case $a in ("$b")
--
Stephane
Re: Effect of double quotes on pathname expansion
Geoff Clare wrote: > Having said that, I now can't find the place in the standard that > covers this. > > Am I just not finding it, or is it genuinely missing? I cannot remember that I did ever find something related in the standard but I am not shure whether it is really missing. BTW: Pathname expansion does not happen because genetic shells keeps double quotes in the parser and remove them in the macro expansion stage that at the same time inserts backslashes to string parts in double quotes or string parts that have been the result of a macro expansion like $a. This is why the following glob stage does not do path name expansion on these parts. The backslashes in question are later removed again in the quote removal stage that is past the path name expansion. P.S.: glob() was previously called gmatch() in SVr4 (and before) based shells. Jörg -- EMail:jo...@schily.net(home) Jörg Schilling D-13353 Berlin joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.org/private/ http://sf.net/projects/schilytools/files/'
Effect of double quotes on pathname expansion
[Starting a new thread for this so it doesn't get lost in the bug 1234 one] Geoff Clare wrote, on 04 Jul 2019: > > Subject: Re: [1003.1(2016)/Issue7+TC2 0001234]: in most shells, backslash > doesn't have two meaning wrt pattern matching > > Harald van Dijk wrote, on 03 Jul 2019: > > > > [...] shell quoting can be used on the asterisk: just try > > > > a='*' > > ls -ld "$a" > > > > This does work for listing only a file literally named '*'. > > That's because of the double quotes around $a. Pathname expansion > is not done inside double quotes, so there is no pattern here, just > a string that contains a '*'. Having said that, I now can't find the place in the standard that covers this. Am I just not finding it, or is it genuinely missing? Note that 2.2.3 Double-Quotes is not sufficient. It says double-quotes preserve the literal value of (certain) characters within the double-quotes. The '*' is not within the double-quotes, so that doesn't apply to it. -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
[1003.1(2016)/Issue7+TC2 0001267]: sh HOME should not be UP shaded
A NOTE has been added to this issue. == http://austingroupbugs.net/view.php?id=1267 == Reported By:dennisw Assigned To: == Project:1003.1(2016)/Issue7+TC2 Issue ID: 1267 Category: Shell and Utilities Type: Error Severity: Comment Priority: normal Status: New Name: Dennis Wölfing Organization: User Reference: Section:sh Page Number:3228 Line Number:108396-108397 Interp Status: --- Final Accepted Text: == Date Submitted: 2019-07-03 15:24 UTC Last Modified: 2019-07-04 08:10 UTC == Summary:sh HOME should not be UP shaded == -- (0004471) geoffclare (manager) - 2019-07-04 08:10 http://austingroupbugs.net/view.php?id=1267#c4471 -- Looking at the history, it seems that this was an editorial mistake when merging POSIX.2-1992 into SUSv2 to make POSIX.1-2001/SUSv3. The text "This volume of IEEE Std 1003.1-2001 specifies the effects of this variable only for systems supporting the User Portability Utilities option" was added to FCEDIT, HISTFILE and HOME but should have been added to FCEDIT, HISTFILE and HISTSIZE (to match POSIX.2-1992). The fix should be to remove UP shading from HOME and add it to HISTSIZE. Issue History Date ModifiedUsername FieldChange == 2019-07-03 15:24 denniswNew Issue 2019-07-03 15:24 denniswName => Dennis Wölfing 2019-07-03 15:24 denniswSection => sh 2019-07-03 15:24 denniswPage Number => 3228 2019-07-03 15:24 denniswLine Number => 108396-108397 2019-07-03 16:22 shware_systems Note Added: 0004470 2019-07-04 08:10 geoffclare Note Added: 0004471 ==
Re: [1003.1(2016)/Issue7+TC2 0001234]: in most shells, backslash doesn't have two meaning wrt pattern matching
Harald van Dijk wrote, on 03 Jul 2019: > > >No, it's a context where shell-quoting backslash *doesn't* work. Therefore > >the backslash should act as an escape character just like in find, pax, > >fnmatch() and glob(). > > It's not about shell quoting the backslash, it's about whether shell quoting > can be used on the asterisk. The quoted sentence says that if shell quoting > cannot be used on the asterisk, then it can be escaped with a backslash > instead. But shell quoting can be used on the asterisk: just try > > a='*' > ls -ld "$a" > > This does work for listing only a file literally named '*'. That's because of the double quotes around $a. Pathname expansion is not done inside double quotes, so there is no pattern here, just a string that contains a '*'. -- Geoff Clare The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
