Stephane Chazelas <[email protected]> wrote, on 09 May 2018:
>
> \ treated specially
> inside bracket expressions in practice is more common than the
> behaviour POSIX specifies where it's not treated specially.
>
> I'm fine that POSIX requires []xyz] to match on ] and [xy-] on -
> and that it's the only portable way to match a ] or - inside a
> bracket expression. I'd much rather use [xy\]z] or [x\-y]
> instead of having to make sure ] is first and - is first or
> last, but I understand some implementations don't support it
> (note the complication with wildcards where you can't do [^!]
> nor [!^] to match on either ^ or !).
>
> I'm not fine with POSIX requiring [\t] to match on backslash and
> t, [\-] to match on backslash and -. Telling application writers
> that they can portably use [\-] to match either \ or - is a lie.
> In practice in many tools, you need [\\-] if not [-\\].
>
> By requiring that, POSIX shot them in the foot. They shut the
> door to possible extensions.
If you want to request changes to the RE rules for backslash, please
submit a separate bug. For bug 1105 we should concentrate on fixing
the lexical treatment of backslash in awk that happens before the RE
rules get involved.
> [...]
> > > Does the change address the differences in behaviour for
> > >
> > > PATTERN='\f' awk '$0 ~ $ENVIRON["PATTERN"]'
> > > or
> > > awk '$0 ~ "\\f"'
> >
> > This was discussed in a conference call and we decided no change
> > was needed as the standard already says the following in the paragraph
> > after the table that the bug resolution modifies:
> >
> > If the right-hand operand is any expression other than the lexical
> > token ERE, the string value of the expression shall be interpreted
> > as an extended regular expression, including the escape conventions
> > described above. Note that these same escape conventions shall also
> > be applied in determining the value of a string literal (the lexical
> > token STRING), and thus shall be applied a second time when a
> > string literal is used in this context.
>
> I don't see the point of "requiring" that $0 ~ "\\f" match a FF.
It's a consequence of the way literal strings are turned into EREs.
When this happens, backslashes are interpreted twice at the lexical
level, as stated clearly in the current standard (in the paragraph
after Table 4-2).
> Many implementations don't do it and I don't imagine them
> changing any time soon (several of them still don't support
> {x,y} RE intervals).
It has been reported to Oracle as a Solaris bug, so that one at
least is expected to change.
Any that don't support {x,y} are hopelessly non-conforming (it's been
a POSIX requirement for 26 years!) which means we shouldn't give them
much, if any, weight when considering other awk conformance matters.
> > > You said earlier that,
> > >
> > > echo b | awk '/[a\55c]/'
> > >
> > > is required to match (on ASCII-based systems), but that's not
> > > the case with several implementations (like nawk on Solaris,
> > > bwk's awk, mawk, FreeBSD awk...).
> > >
> > > Should those be considered non-compliant?
> >
> > The previous discussion was \056 matching any character because it
> > becomes a <period>, but this case should match too (assuming both
> > ASCII encoding and POSIX locale; the latter because it's a range).
> [...]
>
> Note the requirement above that $0 ~ "\\f" match on FF suggests
> an implementation whose regex engine understands "\f", while
> this one suggests those \x being expanded before calling the
> regexp engine.
No. As I mentioned above, in $0 ~ "\\f" the backslashes are
interpreted twice at the lexical level. The regex engine sees an
actual FF character.
> What problem do you see with my suggested resolution (which as
> far as I can tell is compatible with most existing
> implementations)?
The main problem is you didn't supply detailed wording changes.
Another is that there was a major misinterpretation of the standard
in your problem description (about /.../ being like "strong quotes").
So we came up with wording changes ourselves that we believe address
the parts of your problem description that are genuine problems in the
standard.
If you don't like some of our proposed changes, you are free to
suggest alternative detailed wording changes. However, I can't
promise that they will be considered, especially if they take a
different direction than the one we reached consensus on in the calls.
--
Geoff Clare <[email protected]>
The Open Group, Apex Plaza, Forbury Road, Reading, RG1 1AX, England
