[
https://issues.apache.org/jira/browse/GROOVY-12353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Sun updated GROOVY-12353:
--------------------------------
Description:
h3. Problem
After GROOVY-12169 and GROOVY-12171, several everyday syntax mistakes still
produce a generic *Unexpected input* or *Unexpected character* message that
does not name the actual problem.
Invisible or look-alike characters (zero-width space, BOM, NUL, no-break space,
curly quotes, non-ASCII dashes) render as an empty or misleading glyph between
quotes. Unclosed quotes and comments are reported as an unexpected quote or
slash rather than as an unclosed literal. {{if true}} without parentheses, {{x
? y}}, and {{const x = 1}} look like random unexpected tokens instead of a
missing {{(}} / {{:}} or an unimplemented keyword. Groovy 4 safe index is a
single token that pairs with a closing bracket, but an unclosed index inside
parentheses was reported as {{Missing ')'}}.
h3. Examples (before)
* Unclosed string:
{noformat}
println 'Hello
{noformat}
reports:
{noformat}
Unexpected character: ''' @ line 1, column 9.
{noformat}
(caret on the opening quote)
* Unclosed block comment:
{noformat}
/* comment
{noformat}
reports:
{noformat}
Unexpected input: '/'
{noformat}
* Invisible character ({{def}} + U+200B + {{name = null}}):
{noformat}
Unexpected character: ''
{noformat}
* Missing punctuation:
{noformat}
if true { x = 1 }
{noformat}
reports:
{noformat}
Unexpected input: 'true'
{noformat}
* Reserved keyword:
{noformat}
const x = 1
{noformat}
reports:
{noformat}
Unexpected input: 'const'
{noformat}
* Duplicate location:
{noformat}
def n = 1_
{noformat}
reports:
{noformat}
Number ending with underscores is invalid @ line 1, column 10 @ line 1, column
10.
{noformat}
* Hard-coded varargs name:
{noformat}
def m(int... a, int b) {}
{noformat}
reports:
{noformat}
The var-arg parameter strs must be the last parameter
{noformat}
* Unclosed safe index inside parentheses:
{noformat}
(a?[0
{noformat}
reports:
{noformat}
Missing ')'
{noformat}
h3. Root cause
* Lexer {{UNEXPECTED_CHAR}} inlined the raw character with only a quote-escape,
so control, format, and look-alike characters vanish or mislead. An unexpected
quote is almost always an unclosed string, but the message never said so.
* Unclosed block comments failed the comment rule and were retokenized as
{{/}}, so the parser saw an unexpected slash.
* Parser fallback wording was still ANTLR's *Unexpected input*, even when the
expected set was a single punctuation token, or the offender was a reserved or
misplaced keyword ({{const}}, {{goto}}, {{threadsafe}}, {{else}}, {{catch}},
{{finally}}, {{case}}) or EOF. For {{default:}} / {{default ->}} outside a
switch, the offender is {{:}} or {{->}}, not {{default}}.
* The missing-closer diagnostic trusted a sole expected closer. For an unclosed
safe index inside parentheses the parser expected {{)}}, so the inner unclosed
index was reported as {{Missing ')'}}. The safe-index token was not treated as
the same opener family as a normal index.
* Lexer {{require(..., true)}} appended {{@ line N, column M}} to
{{GroovySyntaxError}}, and {{SyntaxException}} appended the same location again.
* {{AstBuilder}} used a hard-coded parameter name {{strs}} in the
varargs-not-last diagnostic.
Grammar-level parser error alternatives are not an option: GROOVY-9588 showed
they enlarge the ATN and slow successful parses. An EOF closer on slashy
strings is also not an option: {{/}} after an expression with newlines is
division, not an unclosed slashy string.
h3. Goal
Give javac-aligned, developer-facing sentences for these common mistakes, with
an accurate caret, without reintroducing parser error alternatives on the hot
path.
h3. Approach
Error-path-only, two layers. Successful parses never enter these helpers.
* Lexer ({{GroovyLexer.g4}} / {{AbstractLexer}}): an unexpected quote becomes
*Unclosed string literal*; an unclosed block comment becomes *Unclosed comment*
at the opener (one non-greedy loop, then the closer or EOF — not a second lexer
rule, which would win by longest match and swallow trailing source). Other
unexpected characters go through {{getCharErrorDisplay}} (shared with the
GString {{$}} path from GROOVY-12171). Unicode spaces other than U+0020, curly
quotes, and non-ASCII dashes are named as a Unicode escape (for example
{{'\u200b'}}) so they do not vanish into the caret line. Stop attaching
position text on lexer {{require}} calls so {{SyntaxException}} is the only
source of {{@ line N, column M}}.
* Parser ({{AbstractFriendlyErrorStrategy}}): {{MissingDelimiterDiagnostic}}
still relocates the caret for a missing closer (GROOVY-12169). Safe index is
the same opener family as a normal index; if the innermost opener is not the
family of the sole expected closer, defer so an unclosed index inside
parentheses reports *Missing ']'*. Everything else only refines the fallback
sentence and keeps ANTLR's offending token: reserved/misplaced keyword, then
{{default:}} / {{default ->}} lookback (skip newlines; do not label an
incomplete interface {{default}} method as outside switch), then a leading
safe-index token without a receiver, then a singleton expected punctuation
token ({{Missing '('}}, {{Missing ':'}}, {{Missing '>'}}, ...), then
*Unexpected end of input* for EOF. Locate/refine stay in a defensive try;
listener dispatch is a single call afterwards.
{{AstBuilder}} reports the actual varargs parameter name.
h3. Expected result (after)
* unclosed single-, double-, or triple-quoted string becomes *Unclosed string
literal*
* unclosed block comment becomes *Unclosed comment* (caret on the opener)
* zero-width space, no-break space, curly quote, em dash become {{Unexpected
character: '\u200b'}} (and the matching escape)
* {{if true}} / {{while true}} / {{for int i in ...}} without parentheses
becomes {{Missing '('}}
* {{x ? y}} becomes {{Missing ':'}}
* a generic type missing {{>}} before {{(}} becomes {{Missing '>'}}
* {{const x = 1}} becomes {{'const' is not supported; use 'val' or 'static
final' instead}}
* {{goto label}} becomes {{'goto' is not supported}}
* {{threadsafe}} as a modifier becomes {{'threadsafe' is not supported}}
* stray {{else}} / {{catch}} / {{case}} become {{'else' without 'if'}} /
{{'catch' without 'try'}} / {{'case' outside of switch}}
* {{default: x}} / {{default -> x}} at script scope becomes {{'default' outside
of switch}}; an incomplete interface {{default}} method is not that message
* unclosed or mismatched safe index, including inside parentheses, becomes
*Missing ']'*
* a safe index with no receiver, or {{?}} immediately before one, becomes:
{noformat}
'?[' requires an expression before it
{noformat}
* a space between the question mark and the opening bracket is an incomplete
ternary ({{Missing ':'}}); safe index is one token:
{noformat}
a? [0]
{noformat}
* {{throw}} at EOF becomes *Unexpected end of input*
* {{def n = 1_}} becomes *Number ending with underscores is invalid* (position
once)
* {{def m(int... a, int b)}} becomes {{The var-arg parameter a must be the last
parameter}}
Valid programs are unchanged.
h3. Related
GROOVY-12169, GROOVY-12171, GROOVY-9588, GROOVY-10146
was:
h3. Problem
After GROOVY-12169 and GROOVY-12171, several everyday syntax mistakes still
produce a generic *Unexpected input* or *Unexpected character* message that
does not name the actual problem.
Invisible characters (zero-width space, BOM, NUL, form feed) appear as an empty
glyph in quotes. Unclosed quotes and comments are reported as an unexpected
quote or slash rather than as an unclosed literal. {{if true}} without
parentheses, {{x ? y}}, and {{const x = 1}} look like random unexpected tokens
instead of a missing {{(}} / {{:}} or an unimplemented keyword.
h3. Examples (before)
* Source:
{noformat}
println 'Hello
{noformat}
Report:
{noformat}
Unexpected character: ''' @ line 1, column 9.
{noformat}
(caret on the opening quote)
* Source:
{noformat}
/* comment
{noformat}
Report:
{noformat}
Unexpected input: '/'
{noformat}
(no mention of an unclosed comment)
* Source: {{def}} + zero-width space + {{name = null}}
Report:
{noformat}
Unexpected character: ''
{noformat}
(the offending character is invisible)
* Source:
{noformat}
if true { x = 1 }
{noformat}
Report:
{noformat}
Unexpected input: 'true'
{noformat}
* Source:
{noformat}
const x = 1
{noformat}
Report:
{noformat}
Unexpected input: 'const'
{noformat}
* Source:
{noformat}
def n = 1_
{noformat}
Report:
{noformat}
Number ending with underscores is invalid @ line 1, column 10 @ line 1, column
10.
{noformat}
(position duplicated)
* Source:
{noformat}
def m(int... a, int b) {}
{noformat}
Report:
{noformat}
The var-arg parameter strs must be the last parameter
{noformat}
(hard-coded name {{strs}})
h3. Root cause
* Lexer {{UNEXPECTED_CHAR}} inlined the raw character with only a
quote-escape, so control / format characters vanish in the message. An
unexpected quote is almost always an unclosed string, but the message never
said so.
* Unclosed block comments failed the comment rule and were retokenised as
{{/}}, so the parser saw an unexpected slash.
* Parser fallback wording was still ANTLR's *Unexpected input*, even when the
expected set was a single punctuation token (open paren, colon, {{>}}) or the
offending token was a reserved keyword ({{const}}, {{goto}}, {{else}},
{{catch}}, {{finally}}, {{case}}) or EOF.
* Lexer {{require(..., true)}} appended {{@ line N, column M}} to
{{GroovySyntaxError}}, and {{SyntaxException}} appended the same location again.
* {{AstBuilder}} used a hard-coded parameter name {{strs}} in the
varargs-not-last diagnostic.
Grammar-level parser error alternatives are not an option: GROOVY-9588 showed
they enlarge the ATN and slow successful parses.
h3. Goal
Give javac-aligned, developer-facing sentences for these common mistakes, with
an accurate caret, without reintroducing parser error alternatives on the hot
path.
h3. Approach
Error-path-only, two layers:
* Lexer ({{GroovyLexer.g4}} / {{AbstractLexer}}): unexpected quote becomes
*Unclosed string literal*; unclosed block comment becomes *Unclosed comment* at
the opener (same-rule EOF alternative, not a second lexer rule); other
unexpected characters via {{getCharErrorDisplay}} (shared with the GString
{{$}} path from GROOVY-12171). Stop attaching position text on lexer
{{require}} calls so {{SyntaxException}} is the only source of {{@ line N,
column M}}.
* Parser ({{AbstractFriendlyErrorStrategy}}): {{MissingDelimiterDiagnostic}}
still relocates the caret for a missing closer (GROOVY-12169). Everything else
only refines the fallback sentence and keeps ANTLR's offending token:
reserved/misplaced keyword, then a singleton expected punctuation token
({{Missing '('}}, {{Missing ':'}}, {{Missing '>'}}, ...), then *Unexpected end
of input* for EOF.
{{AstBuilder}} reports the actual varargs parameter name.
h3. Expected result (after)
* unclosed single-quoted string becomes *Unclosed string literal*
* unclosed block comment becomes *Unclosed comment* (caret on the opener)
* zero-width space in an identifier becomes {{Unexpected character: '\u200b'}}
* {{if true}} without parentheses becomes {{Missing '('}}
* {{x ? y}} becomes {{Missing ':'}}
* a generic type missing {{>}} before {{(}} becomes {{Missing '>'}}
* {{const x = 1}} becomes {{'const' is not supported; use 'val' or 'static
final' instead}}
* {{goto label}} becomes {{'goto' is not supported}}
* stray {{else}} / {{catch}} / {{case}} become {{'else' without 'if'}} /
{{'catch' without 'try'}} / {{'case' outside of switch}}
* {{throw}} at EOF becomes *Unexpected end of input*
* {{def n = 1_}} becomes *Number ending with underscores is invalid* (position
once)
* {{def m(int... a, int b)}} with a later parameter becomes {{The var-arg
parameter a must be the last parameter}}
Valid programs are unchanged. Successful parses never enter these helpers.
h3. Related
GROOVY-12169, GROOVY-12171, GROOVY-9588, GROOVY-10146
> Improve remaining common syntax error messages (unclosed literals, unexpected
> characters, missing punctuation, reserved keywords)
> ---------------------------------------------------------------------------------------------------------------------------------
>
> Key: GROOVY-12353
> URL: https://issues.apache.org/jira/browse/GROOVY-12353
> Project: Groovy
> Issue Type: Improvement
> Reporter: Daniel Sun
> Priority: Major
>
> h3. Problem
> After GROOVY-12169 and GROOVY-12171, several everyday syntax mistakes still
> produce a generic *Unexpected input* or *Unexpected character* message that
> does not name the actual problem.
> Invisible or look-alike characters (zero-width space, BOM, NUL, no-break
> space, curly quotes, non-ASCII dashes) render as an empty or misleading glyph
> between quotes. Unclosed quotes and comments are reported as an unexpected
> quote or slash rather than as an unclosed literal. {{if true}} without
> parentheses, {{x ? y}}, and {{const x = 1}} look like random unexpected
> tokens instead of a missing {{(}} / {{:}} or an unimplemented keyword. Groovy
> 4 safe index is a single token that pairs with a closing bracket, but an
> unclosed index inside parentheses was reported as {{Missing ')'}}.
> h3. Examples (before)
> * Unclosed string:
> {noformat}
> println 'Hello
> {noformat}
> reports:
> {noformat}
> Unexpected character: ''' @ line 1, column 9.
> {noformat}
> (caret on the opening quote)
> * Unclosed block comment:
> {noformat}
> /* comment
> {noformat}
> reports:
> {noformat}
> Unexpected input: '/'
> {noformat}
> * Invisible character ({{def}} + U+200B + {{name = null}}):
> {noformat}
> Unexpected character: ''
> {noformat}
> * Missing punctuation:
> {noformat}
> if true { x = 1 }
> {noformat}
> reports:
> {noformat}
> Unexpected input: 'true'
> {noformat}
> * Reserved keyword:
> {noformat}
> const x = 1
> {noformat}
> reports:
> {noformat}
> Unexpected input: 'const'
> {noformat}
> * Duplicate location:
> {noformat}
> def n = 1_
> {noformat}
> reports:
> {noformat}
> Number ending with underscores is invalid @ line 1, column 10 @ line 1,
> column 10.
> {noformat}
> * Hard-coded varargs name:
> {noformat}
> def m(int... a, int b) {}
> {noformat}
> reports:
> {noformat}
> The var-arg parameter strs must be the last parameter
> {noformat}
> * Unclosed safe index inside parentheses:
> {noformat}
> (a?[0
> {noformat}
> reports:
> {noformat}
> Missing ')'
> {noformat}
> h3. Root cause
> * Lexer {{UNEXPECTED_CHAR}} inlined the raw character with only a
> quote-escape, so control, format, and look-alike characters vanish or
> mislead. An unexpected quote is almost always an unclosed string, but the
> message never said so.
> * Unclosed block comments failed the comment rule and were retokenized as
> {{/}}, so the parser saw an unexpected slash.
> * Parser fallback wording was still ANTLR's *Unexpected input*, even when the
> expected set was a single punctuation token, or the offender was a reserved
> or misplaced keyword ({{const}}, {{goto}}, {{threadsafe}}, {{else}},
> {{catch}}, {{finally}}, {{case}}) or EOF. For {{default:}} / {{default ->}}
> outside a switch, the offender is {{:}} or {{->}}, not {{default}}.
> * The missing-closer diagnostic trusted a sole expected closer. For an
> unclosed safe index inside parentheses the parser expected {{)}}, so the
> inner unclosed index was reported as {{Missing ')'}}. The safe-index token
> was not treated as the same opener family as a normal index.
> * Lexer {{require(..., true)}} appended {{@ line N, column M}} to
> {{GroovySyntaxError}}, and {{SyntaxException}} appended the same location
> again.
> * {{AstBuilder}} used a hard-coded parameter name {{strs}} in the
> varargs-not-last diagnostic.
> Grammar-level parser error alternatives are not an option: GROOVY-9588 showed
> they enlarge the ATN and slow successful parses. An EOF closer on slashy
> strings is also not an option: {{/}} after an expression with newlines is
> division, not an unclosed slashy string.
> h3. Goal
> Give javac-aligned, developer-facing sentences for these common mistakes,
> with an accurate caret, without reintroducing parser error alternatives on
> the hot path.
> h3. Approach
> Error-path-only, two layers. Successful parses never enter these helpers.
> * Lexer ({{GroovyLexer.g4}} / {{AbstractLexer}}): an unexpected quote becomes
> *Unclosed string literal*; an unclosed block comment becomes *Unclosed
> comment* at the opener (one non-greedy loop, then the closer or EOF — not a
> second lexer rule, which would win by longest match and swallow trailing
> source). Other unexpected characters go through {{getCharErrorDisplay}}
> (shared with the GString {{$}} path from GROOVY-12171). Unicode spaces other
> than U+0020, curly quotes, and non-ASCII dashes are named as a Unicode escape
> (for example {{'\u200b'}}) so they do not vanish into the caret line. Stop
> attaching position text on lexer {{require}} calls so {{SyntaxException}} is
> the only source of {{@ line N, column M}}.
> * Parser ({{AbstractFriendlyErrorStrategy}}): {{MissingDelimiterDiagnostic}}
> still relocates the caret for a missing closer (GROOVY-12169). Safe index is
> the same opener family as a normal index; if the innermost opener is not the
> family of the sole expected closer, defer so an unclosed index inside
> parentheses reports *Missing ']'*. Everything else only refines the fallback
> sentence and keeps ANTLR's offending token: reserved/misplaced keyword, then
> {{default:}} / {{default ->}} lookback (skip newlines; do not label an
> incomplete interface {{default}} method as outside switch), then a leading
> safe-index token without a receiver, then a singleton expected punctuation
> token ({{Missing '('}}, {{Missing ':'}}, {{Missing '>'}}, ...), then
> *Unexpected end of input* for EOF. Locate/refine stay in a defensive try;
> listener dispatch is a single call afterwards.
> {{AstBuilder}} reports the actual varargs parameter name.
> h3. Expected result (after)
> * unclosed single-, double-, or triple-quoted string becomes *Unclosed string
> literal*
> * unclosed block comment becomes *Unclosed comment* (caret on the opener)
> * zero-width space, no-break space, curly quote, em dash become {{Unexpected
> character: '\u200b'}} (and the matching escape)
> * {{if true}} / {{while true}} / {{for int i in ...}} without parentheses
> becomes {{Missing '('}}
> * {{x ? y}} becomes {{Missing ':'}}
> * a generic type missing {{>}} before {{(}} becomes {{Missing '>'}}
> * {{const x = 1}} becomes {{'const' is not supported; use 'val' or 'static
> final' instead}}
> * {{goto label}} becomes {{'goto' is not supported}}
> * {{threadsafe}} as a modifier becomes {{'threadsafe' is not supported}}
> * stray {{else}} / {{catch}} / {{case}} become {{'else' without 'if'}} /
> {{'catch' without 'try'}} / {{'case' outside of switch}}
> * {{default: x}} / {{default -> x}} at script scope becomes {{'default'
> outside of switch}}; an incomplete interface {{default}} method is not that
> message
> * unclosed or mismatched safe index, including inside parentheses, becomes
> *Missing ']'*
> * a safe index with no receiver, or {{?}} immediately before one, becomes:
> {noformat}
> '?[' requires an expression before it
> {noformat}
> * a space between the question mark and the opening bracket is an incomplete
> ternary ({{Missing ':'}}); safe index is one token:
> {noformat}
> a? [0]
> {noformat}
> * {{throw}} at EOF becomes *Unexpected end of input*
> * {{def n = 1_}} becomes *Number ending with underscores is invalid*
> (position once)
> * {{def m(int... a, int b)}} becomes {{The var-arg parameter a must be the
> last parameter}}
> Valid programs are unchanged.
> h3. Related
> GROOVY-12169, GROOVY-12171, GROOVY-9588, GROOVY-10146
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
