[
https://issues.apache.org/jira/browse/GROOVY-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18130108#comment-18130108
]
Paul King commented on GROOVY-12435:
------------------------------------
AI read:
GROOVY-12435 is confirmed. It has been there since the antlr4 ("Parrot") parser
arrived in Groovy 3.0, and it's a regression compared with the antlr2 parser.
Wherever a line ends in a lone `\r`, line and column numbers after it are
wrong, and stack traces and compile errors point at the wrong place.
## Reproduction
I rebuilt the attachment from its byte count rather than reading it directly,
because the sandbox wouldn't let me read the saved copy. The key byte is a lone
`\r` right after `package foo`; the rest uses CRLF.
| Case | master, 5.0.6, 4.0.32, 3.0.25
(antlr4) | 3.0.25 antlr2 | Correct |
| -------------------------------------------- |
-------------------------------------- | ------------- | ------- |
| Issue shape: import, statement | lines 2, 6
| 3, 7 | 3, 7 |
| All-CR file, 3 statements | lines 1, 1, 1; columns 1, 11,
22 | 1, 2, 4 | 1, 2, 4 |
| CRs inside a `/* */` comment, next statement | line 3
| 5 | 5 |
| CR inside a `'''` string, next statement | line 2
| 3 | 3 |
| Runtime stack trace for a throw on line 3 | line 1
| 3 | 3 |
| Syntax error on line 3, column 9 | line 1, column 29
| 3:9 | 3:9 |
Master gives exactly the reporter's numbers (2 and 6).
## Cause
- The grammar is fine: `LineTerminator : '\r'? '\n' | '\r'`, so a lone `\r`
becomes a normal newline token.
- Line counting happens in the ANTLR runtime, which Groovy uses as the
`me.sunlan:antlr4-runtime` 4.13.2.17 fork. Its
`LexerATNSimulator.consume(input, curChar)` only does `line++` on `'\n'`.
That's the upstream ANTLR behaviour Eric's comment links to.
- In the fork, that method is `final` and package-private, and the main DFA
loop calls it directly. So overriding the public `consume(input)` in Groovy's
existing `PositionAdjustingLexerATNSimulator` (in `GroovyLangLexer.java`)
wouldn't help.
- A second, smaller bug: `PositionConfigureUtils.java:65` works out a node's
end line by counting only `'\n'` in the last token. A multi-line string with
lone CRs would still get a wrong end line even after the lexer is fixed.
- The rest of Groovy disagrees with the parser. `AbstractReaderSource.getLine`
uses `BufferedReader.readLine()`, which does treat a lone `\r` as a line end.
So today an error message's source excerpt shows a different line from the line
number it reports.
## Fix options
**A. Fix the fork (cleanest).** In `consume(input, curChar)`, also count `'\r'`
as a new line when `input.LA(2) != '\n'`. The extra look-ahead only happens on
`\r` characters, so it costs essentially nothing. It needs a new fork release
from Daniel, a version bump in `versions.properties`, and regenerated
`verification-metadata.xml`. It deliberately departs from upstream ANTLR, which
the fork already does for performance.
**B. Groovy-only.** Override `match(CharStream, int mode)` in
`PositionAdjustingLexerATNSimulator`. After `super.match`, rescan the token
just read for lone `\r` and correct `line` and `charPositionInLine`.
- `match` runs for every token, including skipped whitespace and comments, so
start positions come out right.
- To keep the lexer's hot path untouched, check once per file whether it
contains a lone `\r`, and skip the rescan entirely when it doesn't.
- No dependency release is needed, and it can be backported to 4.x and 5.x
unchanged.
Either way, `PositionConfigureUtils` needs the matching change: count `\r\n`, a
lone `\r`, and a lone `\n` each as one line end. Tests can follow the table:
AST line and column numbers, a stack-trace line, and a syntax-error position,
for lone-CR, mixed and all-CR files.
**Recommendation:** use B for 6.0.x and the backports, since it fixes this with
no dependency change. Ask Daniel whether he'd take A in the fork; if he does, B
can be removed later. This isn't a release blocker: lone-CR endings are rare
(old Mac files and mixed-ending files from editors). But it's a real tooling
problem, as the reporter found in his editor, and the stack-trace line numbers
are wrong too.
> carriage-return only line ending does not increment line number
> ---------------------------------------------------------------
>
> Key: GROOVY-12435
> URL: https://issues.apache.org/jira/browse/GROOVY-12435
> Project: Groovy
> Issue Type: Bug
> Components: parser-antlr4
> Affects Versions: 3.0.25, 4.0.33, 5.1.3, 6.0.0
> Reporter: Eric Milles
> Priority: Major
> Attachments: LineEndings.groovy, image-2026-09-30-09-41-06-973.png
>
>
> [^LineEndings.groovy]I had a source file with mixed line endings that caused
> a lot of trouble with the editor. I traced this back to the parser's handling
> of line endings. It seems when a line ending is just a carriage-return (\r)
> the line number is not incremented.
> !image-2026-09-30-09-41-06-973.png!
> In this example, the import and script statement have lines 2 and 6
> respectively. They should be at lines 3 and 7 as can be seen by the line
> numbering in the editor's gutter. The antlr2 parser handled this case
> properly.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
