[
https://issues.apache.org/jira/browse/GROOVY-12398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18114300#comment-18114300
]
ASF GitHub Bot commented on GROOVY-12398:
-----------------------------------------
daniellansun commented on PR #2921:
URL: https://github.com/apache/groovy/pull/2921#issuecomment-5637141960
Thanks Paul — that's a much closer reading than mine was, and I agree with
the recommendation.
Master only for now; I would not put this in the 6.0 RC line. The 1.6×
figure is parse-only, and a grammar/ATN change is the wrong kind of surprise
during RC. A backport after some settling time (and a corpus AST sweep if we do
go there) sounds right.
On the three deltas:
1. **Supplementary-plane capitalization.** You are right that
`isUpperCase(LA(-1))` after a surrogate pair only saw the low surrogate, so
everything in that plane used to lex as `Identifier`. Using the decoded code
point is a real parse change for `className`. I think the new behaviour is the
one we want, but it is not “strictly performance”. I have extracted
`lastConsumedCodePoint()` so that mistake is harder to reintroduce, and added
tests (Deseret capital vs small, type parameter vs command,
`isInvalidLocalVariableDeclaration` on the code point). Follow-up: `e27a4dabbd`.
2. **`elementValues` gate.** Glad that checked out. I added a sweep over
`identifier ∪ keywords` (including `MODULE`) so the BitSet cannot drift
silently, plus a note on those two parser rules.
3. **Invalid-octal caret.** Agreed it was a latent counter bug. The
follow-up reports from the token start (same pattern as unclosed comments), so
a suffix or a second `08` in the file cannot shift the caret. Tests cover `08`,
`08L`, `08 09`, and `errorIgnored`.
Happy to drop or reword any of that if you would rather keep the old
supplementary-plane classification for this PR and treat the code-point fix
separately.
> Improve lexer and parser performance
> ------------------------------------
>
> Key: GROOVY-12398
> URL: https://issues.apache.org/jira/browse/GROOVY-12398
> Project: Groovy
> Issue Type: Improvement
> Reporter: Daniel Sun
> Priority: Major
>
> The Groovy ANTLR4 grammar already uses left-recursive expressions, which is
> the main idea of the [optimized Java
> grammar|https://github.com/antlr/grammars-v4/tree/master/java/java]. Several
> remaining lexer and parser costs still match patterns that grammar was
> written to avoid.
> h3. Problem
> * Number literals are assembled from a chain of one-character fragments
> ({{Digit}}, {{Zero}}, {{NonZeroDigit}}, ...). The invalid-octal alternative
> also counts digits in a mid-rule action loop, which prevents DFA compilation
> of {{IntegerLiteral}}.
> * Every identifier, including ASCII names such as {{Foo}} and {{foo}},
> evaluates {{Character.isUpperCase}} to choose {{CapitalizedIdentifier}} vs
> {{Identifier}}.
> * GString and comment predicates ({{isFollowedByJavaLetterInGString}},
> {{isFollowedByWhiteSpaces}}) allocate a one-character {{String}} and run
> {{Pattern.matcher}} on each lookahead.
> * {{elementValues}} is ambiguous with assignment: {{@Foo(a = 1)}} can be
> named pairs or an assignment expression, so {{AdaptivePredict}} explores both
> (see [antlr/grammars-v4#215|https://github.com/antlr/grammars-v4/issues/215]).
> h3. Approach
> Keep token types and language semantics unchanged.
> * Flatten number and float fragments to character classes, as in the
> optimized Java lexer. Validate invalid octal once at the end of the
> alternative.
> * Split ASCII identifiers on a predicate-free DFA path: uppercase {{A-Z}}
> becomes {{CapitalizedIdentifier}}; {{a-z}}, dollar and underscore become
> {{Identifier}}. Unicode still uses predicates.
> * Replace the regex-based predicates with direct character tests.
> * Gate {{elementValue}} with {{!isIdentifierAssign(_input)}}, the same idea
> as Java's {{IsNotIdentifierAssign}}.
> h3. Result
> Local parse-only measurement (SLL then LL) of {{src/main/groovy}} +
> {{src/test-resources/core}} (245 files, approximately 570k characters):
> ||Run||Best||Mean||
> |before|0.231s|0.401s|
> |after|0.146s|0.169s|
> Best run is about 1.6x faster. Mean improves more because prediction is more
> stable after DFA warmup.
> Related: GROOVY-10454, GROOVY-9588
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
