[ 
https://issues.apache.org/jira/browse/TEXT-209?focusedWorklogId=772342&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-772342
 ]

ASF GitHub Bot logged work on TEXT-209:
---------------------------------------

                Author: ASF GitHub Bot
            Created on: 19/May/22 09:26
            Start Date: 19/May/22 09:26
    Worklog Time Spent: 10m 
      Work Description: kinow commented on PR #324:
URL: https://github.com/apache/commons-text/pull/324#issuecomment-1131459543

   Merged. Thanks!!




Issue Time Tracking
-------------------

    Worklog Id:     (was: 772342)
    Time Spent: 0.5h  (was: 20m)

> LookupTranslator returns count of chars consumed, not of codepoints consumed
> ----------------------------------------------------------------------------
>
>                 Key: TEXT-209
>                 URL: https://issues.apache.org/jira/browse/TEXT-209
>             Project: Commons Text
>          Issue Type: Bug
>    Affects Versions: 1.9
>            Reporter: K P
>            Assignee: Bruno P. Kinoshita
>            Priority: Minor
>              Labels: Surrogates, Unicode
>             Fix For: 1.10
>
>          Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> {{{}The contract of abstract method 
> [translate|https://github.com/apache/commons-text/blob/c7361037993be487e0220069d76caf9f3698e2e3/src/main/java/org/apache/commons/text/translate/CharSequenceTranslator.java#L83]
>  i{}}}{{{}n the class 
> [CharSequenceTranslator|https://github.com/apache/commons-text/blob/c7361037993be487e0220069d76caf9f3698e2e3/src/main/java/org/apache/commons/text/translate/CharSequenceTranslator.java]{}}}{{{},
>  and therefore also in the inherited 
> [LookupTranslator|https://github.com/apache/commons-text/blob/c7361037993be487e0220069d76caf9f3698e2e3/src/main/java/org/apache/commons/text/translate/LookupTranslator.java],
>  is to return the "{_}int count of codepoints consumed{_}".{}}}
> Cf. their javadoc.
> However, LookupTranslator returns the number of chars.
> This can be seen in its source, in its implementation of the abstract method, 
> where it returns "i", which is the length _in chars_ of the longest matching 
> substring.
> Test to reproduce:
> Define a mapping where a String with 1 supplementary character is mapped to 1 
> (basic) char.
> {code:java}
> /* Key: string with Mathematical double-struck capital A (U+1D538) */
> String symbol = new StringBuilder().appendCodePoint(0x1D538).toString();
> /* Map U+1D538 to "A" */
> Map<CharSequence, CharSequence> map = new HashMap<>();
> map.put(symbol, "A");
> LookupTranslator translator = new LookupTranslator(map);
> String translated = translator.translate(symbol + "=A");
>               
> /* Fails: instead of "A=A", we get "AA". */
> assertEquals("A=A", translated);
> {code}
> So when doing the translation, the supplementary character got mapped, but 
> then you notice that the LookupTranslator erroneously {+}_swallowed_ the 
> following "=" character{+}.
> That is because its translate method returns the count of matched _chars_ 
> (i.e. 2 = the high and low surrogate code unit  (chars) that form the 
> surrogate pair)  , instead of the count of matched _codepoints_ (i.e. which 
> is 1, and which the javadoc claims to return)



--
This message was sent by Atlassian Jira
(v8.20.7#820007)

Reply via email to