[jira] [Commented] (GROOVY-12085) SourceText slices UTF-16 with code-point AST columns, truncating source for supplementary characters

[ASF GitHub Bot (Jira)]

    [ 
https://issues.apache.org/jira/browse/GROOVY-12085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18088663#comment-18088663
 ] 

ASF GitHub Bot commented on GROOVY-12085:
-----------------------------------------

testlens-app[bot] commented on PR #2606:
URL: https://github.com/apache/groovy/pull/2606#issuecomment-4697080168

   ## 🚨 TestLens detected 1 failed test 🚨
   
   Here is what you can do:
   
   1) Inspect the test failures carefully.
   2) If you are convinced that some of the tests are flaky, you can mute them 
below.
   3) Finally, trigger a rerun by checking the rerun checkbox.
   
   ### Test Summary
   
   | Check | Project/Task | Test | Runs |
   |---|---|---|---|
   | [Build and test / lts \(17, windows-latest, 
1\)](https://github.com/apache/groovy/actions/runs/27452890775/job/81151957551?pr=2606)
 | :test | Groovy11046 > testMissingDependency2\(\) | ❌ |
   
   🏷️ Commit: d65709813bf4b47ec05989d6172622d08fb8a4a2
   ▶️ Tests:  21215 executed
   🟡 Checks: 4/28 completed
   
   ### Test Failures
   
   <details>
   
   <summary><strong>Groovy11046 > testMissingDependency2()</strong> (:test in 
<a 
href="https://github.com/apache/groovy/actions/runs/27452890775/job/81151957551?pr=2606";>Build
 and test / lts (17, windows-latest, 1)</a>)</summary>
   
   ```
   java.lang.AssertionError: script should have failed at runtime. Expression: 
(err.asString() =~ at org.apache.logging.log4j.LogManager.getLogger\()
        at 
org.codehaus.groovy.runtime.InvokerHelper.createAssertError(InvokerHelper.java:402)
        at bugs.Groovy11046.testMissingDependency2(Groovy11046.groovy:52)
   ```
   
   </details>
   
   ### Muted Tests
   > [!NOTE]
   > Checks are currently running using the configuration below.
   
   Select tests to mute in this pull request:
   
   🔲 Groovy11046 > testMissingDependency2\(\) <!

> SourceText slices UTF-16 with code-point AST columns, truncating source for 
> supplementary characters
> ----------------------------------------------------------------------------------------------------
>
>                 Key: GROOVY-12085
>                 URL: https://issues.apache.org/jira/browse/GROOVY-12085
>             Project: Groovy
>          Issue Type: Bug
>            Reporter: Paul King
>            Assignee: Paul King
>            Priority: Major
>              Labels: breaking
>
> Power-assert renders truncated source text for any assertion containing 
> supplementary (astral-plane) characters such as emoji. Anything reusing 
> {{SourceText}} (e.g. groovy-contracts/groovy-verify source capture) is 
> affected, and the same column mismatch mis-positions syntax-error carets.
> h4. Steps to reproduce
> {code:groovy}
> assert (true ? '🥤🐝' : 'z').length() == 999
> {code}
> The rendered expression is cut short by one character per emoji preceding the 
> end (e.g. {{== 999}} renders as {{== 9}}).
> h4. Root cause
> AST column numbers are *code-point*-based ({{GroovyLangLexer}} uses 
> {{CharStreams.fromReader}} -> ANTLR {{CodePointCharStream}}; 
> {{PositionConfigureUtils}} sets {{columnNumber = getCharPositionInLine() + 
> 1}}), but {{SourceText}} slices a *UTF-16* {{String}} from 
> {{SourceUnit.getSample()}} with {{substring()}} using those columns. Each 
> astral char before the slice boundary under-counts the UTF-16 index by one, 
> cutting the slice short.
> Compounding this, {{PositionConfigureUtils}} computes {{lastColumnNumber = 
> getCharPositionInLine() + 1 + token.getText().length()}} -- a code-point 
> start plus a UTF-16 length. The UTF-16 term accidentally compensates for 
> astral chars *inside* the final token (so "emoji as last token" works) but 
> not for those *before* it, which is why the bug looks intermittent.
> h4. Suggested fix
> Make column numbers uniformly code-point-based 
> ({{token.getText().codePointCount(0, len)}} in {{PositionConfigureUtils}}), 
> then convert code-point columns to UTF-16 indices at the slice sites via 
> {{String.offsetByCodePoints}} ({{SourceText}}, and the syntax-error message 
> renderers). Note: converting the hybrid {{lastColumnNumber}} without the 
> {{codePointCount}} fix first will overshoot/throw, so both changes are needed 
> together.
> h4. Affects
> All current versions (present on master). Affects power-assert rendering and 
> any consumer of {{SourceText}}/AST column positions for supplementary 
> characters.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)