[1003.1(2008)/Issue 7 0000375]: Extend test/[...] conditionals: ==, <, >, -nt, -ot, -ef
A NOTE has been added to this issue. == http://austingroupbugs.net/view.php?id=375 == Reported By:dwheeler Assigned To:ajosey == Project:1003.1(2008)/Issue 7 Issue ID: 375 Category: Shell and Utilities Type: Enhancement Request Severity: Objection Priority: normal Status: Under Review Name: David A. Wheeler Organization: User Reference: Section:test Page Number:3224-3225 Line Number:107503-107513 Interp Status: --- Final Accepted Text: == Date Submitted: 2011-02-07 18:34 UTC Last Modified: 2016-09-21 22:44 UTC == Summary:Extend test/[...] conditionals: ==, <, >, -nt, -ot, -ef == Relationships ID Summary -- has duplicate 762 Add == as synonym for related to 813 Utility numeric argument syntax require... == -- (0003388) shware_systems (reporter) - 2016-09-21 22:44 http://austingroupbugs.net/view.php?id=375#c3388 -- This proposal, as a white paper, is a case where the behaviors are being merged to be future compatible more than backwards. This is also why test is being deprecated and marked LEGACY. Both it and the [[ extensions are broken one way or the other, so this proposal is taking what isn't broken and giving it a firmer logical model by adding it to the grammar with extensibility provisions. The caveat about what the standard says being reliable only in the C locale and locales using nationalized ASCII applies here too, for now; future Enhancement Reqs. against the V8 drafts that define additional required locales will address remaining localization concerns. It is known this may break any current script using [[ as an extension. The changes to fix those scripts are expected to be minimal; as proposed, for EREs, enclosing the entire WORD in double quotes will suffice in most cases, only needing \", \` and \$ internally to disable substitutions processing. If tilde expansion is enabled and desired something like "head"~"tail" is required. The onus is already on script authors, if they don't enclose a WORD in quotes, to escape any character that implicitly delimits the token. New scripts are expected to do version checking that they're running on V8 or later to avoid conflicts of interpretation with older shells. It should not break any conforming scripts that were only using test with operators other than -a or -o. Last, for portable scripts blank detection is currently not locale-dependant. For tokenization only the SPC and TAB code points count, as the default members of the blank charclass. Recognizing additional members of a locale's blank charclass is undefined behavior. Adding default members from ISO-646 is possible, but not likely. Issue History Date ModifiedUsername FieldChange == 2011-02-07 18:34 dwheeler New Issue 2011-02-07 18:34 dwheeler Status New => Under Review 2011-02-07 18:34 dwheeler Assigned To => ajosey 2011-02-07 18:34 dwheeler Name => David A. Wheeler 2011-02-07 18:34 dwheeler Section => test 2011-02-07 18:34 dwheeler Page Number => 3224-3225 2011-02-07 18:34 dwheeler Line Number => 107503-107513 2011-02-07 18:49 dwheeler Note Added: 666 2011-02-07 20:23 Don Cragun Interp Status => --- 2011-02-07 20:23 Don Cragun Note Added: 667 2011-02-07 20:23 Don Cragun Severity Editorial => Objection 2011-02-07 20:23 Don Cragun Type Clarification Requested => Enhancement Request 2011-02-07 21:56 dwheeler Note Added: 668 2011-02-07 22:20 Don Cragun Description Updated 2011-02-07 22:20 Don Cragun Desired Action
[1003.1(2008)/Issue 7 0000375]: Extend test/[...] conditionals: ==, <, >, -nt, -ot, -ef
A NOTE has been added to this issue.
==
http://austingroupbugs.net/view.php?id=375
==
Reported By:dwheeler
Assigned To:ajosey
==
Project:1003.1(2008)/Issue 7
Issue ID: 375
Category: Shell and Utilities
Type: Enhancement Request
Severity: Objection
Priority: normal
Status: Under Review
Name: David A. Wheeler
Organization:
User Reference:
Section:test
Page Number:3224-3225
Line Number:107503-107513
Interp Status: ---
Final Accepted Text:
==
Date Submitted: 2011-02-07 18:34 UTC
Last Modified: 2016-09-21 10:20 UTC
==
Summary:Extend test/[...] conditionals: ==, <, >, -nt, -ot,
-ef
==
Relationships ID Summary
--
has duplicate 762 Add == as synonym for
related to 813 Utility numeric argument syntax require...
==
--
(0003387) stephane (reporter) - 2016-09-21 10:20
http://austingroupbugs.net/view.php?id=375#c3387
--
Let me rephrase. [[ =~ ]] in ksh93 and bash32+ is a bit broken by design,
but to use it properly in both (which seems to be the intent of this
proposal: to specify an operator compatible with both), it seems we need
(for the argument after =~) to:
- quote &, <, >, blanks (locale-dependant (*)) and unmatched ) and some
unmatched ] at least or we'd get errors in bash or ksh93
- *not* quote the ERE operators (.*?+[]{}()|$^) if we want them to retain
their special ERE significance. You'll notice that several of those (()|)
can't be in WORD
- quote `, ~, $ and quotes to remove their special meaning as shell tokens
(but not inside [...] for some)
- to remove the special meaning of those ERE operators, we can use double
quotes, single quotes, backslash or $'...' but not when they're inside
[...].
- other characters can be quoted as well but not with backslash as that
could introduce ERE extensions like \<, \>, \b, \w in ksh93.
- for parts of the arguments that are the result of an unquoted expansion
other than ~ expansion, \ (in the content of the expansion) removes the
special meaning of ERE operators and may introduce new ones.
Examples:
a='' bash -c '[[ $a =~ <.*> ]]' gives an error
a='foo' ksh93 -c '[[ $a =~ \<.*\> ]]' returns true (\<, \> taken as word
boundaries)
You need to use:
a='' shell -c '[[ $a =~ "<".*">" ]]' so it works in both shells or
a='' shell -c 're="<.*>"; [[ $a =~ $re ]]' which also works in zsh
and bash31
Other example:
a='blah' shell -c '[[ $a =~ [xy)] ]]' gives an error in both ksh and
bash
a='\' shell -c '[[ $a =~ [xy\)] ]]' doesn't give an error but matches in
ksh (not in bash). and [[ $a =~ [xy")"] ]] also matches on backslash in
ksh93, bash used to have a similar bug.
a='blah' shell -c 're="[xy)]"; [[ $a =~ $re ]]' works in all shells (zsh,
bash31 included)
(*) as already discussed, since blank recognition is locale dependant, you
get behaviours like:
$ LC_CTYPE=fr_FR.ISO8859-15@euro bash -c '[[ $a =~ tête-à-tête ]]'
bash: -c: line 0: syntax error in conditional expression
bash: -c: line 0: syntax error near `tête-à-tête'
bash: -c: line 0: `[[ $a =~ tête-à-tête ]]'
as that à UTF-8 character is 0xc3 0xa0 and 0xa0 happens to be a blank in
ISO8859-15 locales on Solaris.
So in effect you may need to quote every character that is not in the
portable character set in case they may be a blank in the user's locale.
Issue History
Date ModifiedUsername FieldChange
==
2011-02-07 18:34 dwheeler New Issue
2011-02-07 18:34 dwheeler Status New => Under Review
2011-02-07 18:34 dwheeler Assigned To => ajosey
2011-02-07 18:34 dwheeler Name => David A. Wheeler
2011-02-07 18:34 dwheeler Section => test
2011-02-07 18:34 dwheeler Page Number => 3224-3225
2011-02-07 18:34 dwheeler Line Number =>
[1003.1(2008)/Issue 7 0000375]: Extend test/[...] conditionals: ==, <, >, -nt, -ot, -ef
A NOTE has been added to this issue. == http://austingroupbugs.net/view.php?id=375 == Reported By:dwheeler Assigned To:ajosey == Project:1003.1(2008)/Issue 7 Issue ID: 375 Category: Shell and Utilities Type: Enhancement Request Severity: Objection Priority: normal Status: Under Review Name: David A. Wheeler Organization: User Reference: Section:test Page Number:3224-3225 Line Number:107503-107513 Interp Status: --- Final Accepted Text: == Date Submitted: 2011-02-07 18:34 UTC Last Modified: 2016-09-21 07:35 UTC == Summary:Extend test/[...] conditionals: ==, <, >, -nt, -ot, -ef == Relationships ID Summary -- has duplicate 762 Add == as synonym for related to 813 Utility numeric argument syntax require... == -- (0003386) shware_systems (reporter) - 2016-09-21 07:35 http://austingroupbugs.net/view.php?id=375#c3386 -- Re: 3384 Ok, I oversimplified a bit on 1. It is the result of the WORD, after substitutions and quote removal, but minus field splitting, globbing, and I was thinking tilde expansions, that is meant to be reparsed as an extended_reg_exp. It is left to the script author to figure out how much the input needs to be quoted so the result has blanks and other meta-characters EREs don't consider special appearing where required after the quote removal. I still think an additional token type isn't required for expressing this in the grammar. Possibly a Note 11 if this is cleaner than adding to Note 10, but the overall context is more similar to the pattern production than the context of ASSIGNMENT_WORD, that I see. If tilde expansion is included then 2. should return true, otherwise no. I can see limiting tilde expansion to those operators that expect a filepath string as the WORD result, and non-special otherwise. Issue History Date ModifiedUsername FieldChange == 2011-02-07 18:34 dwheeler New Issue 2011-02-07 18:34 dwheeler Status New => Under Review 2011-02-07 18:34 dwheeler Assigned To => ajosey 2011-02-07 18:34 dwheeler Name => David A. Wheeler 2011-02-07 18:34 dwheeler Section => test 2011-02-07 18:34 dwheeler Page Number => 3224-3225 2011-02-07 18:34 dwheeler Line Number => 107503-107513 2011-02-07 18:49 dwheeler Note Added: 666 2011-02-07 20:23 Don Cragun Interp Status => --- 2011-02-07 20:23 Don Cragun Note Added: 667 2011-02-07 20:23 Don Cragun Severity Editorial => Objection 2011-02-07 20:23 Don Cragun Type Clarification Requested => Enhancement Request 2011-02-07 21:56 dwheeler Note Added: 668 2011-02-07 22:20 Don Cragun Description Updated 2011-02-07 22:20 Don Cragun Desired Action Updated 2011-02-07 22:22 Don Cragun Note Edited: 667 2011-02-07 22:50 dwheeler Note Added: 669 2011-02-07 23:13 eblake Note Added: 670 2011-03-08 03:15 dwheeler Note Added: 688 2011-04-24 13:16 jsonn Note Added: 754 2011-04-24 19:01 dwheeler Note Added: 755 2011-04-24 21:48 jsonn Note Added: 756 2011-04-25 20:45 dwheeler Note Added: 758 2011-04-26 09:25 markh Note Added: 759 2011-04-26 09:57 wpollock Note Added: 760 2011-04-26 10:36 jsonn
