[jira] [Updated] (SPARK-49204) Handle surrogate pairs properly

[Jira]

     [ 
https://issues.apache.org/jira/browse/SPARK-49204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uroš Bojanić updated SPARK-49204:
---------------------------------
    Description: 
Fix the following string expressions to handle one-to-many case mapping 
properly:
 * StringReplace
 * StringInstr
 * StringLocate
 * SubstringIndex
 * StringTrim
 * StringTrimLeft
 * StringTrimRight

 

Examples of incorrect results (under {{ICU}} collations):
{code:java}
StringReplace("😄a", "a", "b") // returns: "😄ab" (incorrect), instead of: "😄b" 
(correct){code}
 
{code:java}
StringInstr("😄a", "a") // returns: 3 (incorrect), instead of: 2 (correct){code}
 
{code:java}
StringLocate("a", "😄a") // returns: 3 (incorrect), instead of: 2 (correct){code}
 
{code:java}
SubstringIndex("😄a", "a") // returns: "😄a" (incorrect), instead of: "😄" 
(correct){code}
 
{code:java}
StringTrim("😄", "😄") // returns: "😄" (incorrect), instead of: "" (correct){code}
 
{code:java}
StringTrimLeft("😄", "😄") // returns: "😄" (incorrect), instead of: "" 
(correct){code}
 
{code:java}
StringTrimRight("😄", "😄") // returns: "😄" (incorrect), instead of: "" 
(correct){code}

  was:
Fix the following string expressions to handle one-to-many case mapping 
properly:
 * StringReplace
 * StringInstr
 * StringLocate
 * SubstringIndex
 * StringTrim
 * StringTrimLeft
 * StringTrimRight

 

Examples of incorrect results (under {{ICU}} collations):
{code:java}
StringReplace("🙃a", "a", "b") // returns: "🙃ab" (incorrect), instead of: "🙃b" 
(correct){code}
 

 

 


> Handle surrogate pairs properly
> -------------------------------
>
>                 Key: SPARK-49204
>                 URL: https://issues.apache.org/jira/browse/SPARK-49204
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 4.0.0
>            Reporter: Uroš Bojanić
>            Priority: Major
>              Labels: pull-request-available
>
> Fix the following string expressions to handle one-to-many case mapping 
> properly:
>  * StringReplace
>  * StringInstr
>  * StringLocate
>  * SubstringIndex
>  * StringTrim
>  * StringTrimLeft
>  * StringTrimRight
>  
> Examples of incorrect results (under {{ICU}} collations):
> {code:java}
> StringReplace("😄a", "a", "b") // returns: "😄ab" (incorrect), instead of: "😄b" 
> (correct){code}
>  
> {code:java}
> StringInstr("😄a", "a") // returns: 3 (incorrect), instead of: 2 
> (correct){code}
>  
> {code:java}
> StringLocate("a", "😄a") // returns: 3 (incorrect), instead of: 2 
> (correct){code}
>  
> {code:java}
> SubstringIndex("😄a", "a") // returns: "😄a" (incorrect), instead of: "😄" 
> (correct){code}
>  
> {code:java}
> StringTrim("😄", "😄") // returns: "😄" (incorrect), instead of: "" 
> (correct){code}
>  
> {code:java}
> StringTrimLeft("😄", "😄") // returns: "😄" (incorrect), instead of: "" 
> (correct){code}
>  
> {code:java}
> StringTrimRight("😄", "😄") // returns: "😄" (incorrect), instead of: "" 
> (correct){code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org