[
https://issues.apache.org/jira/browse/SPARK-49204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uroš Bojanić updated SPARK-49204:
---------------------------------
Description:
Fix the following string expressions to handle one-to-many case mapping
properly:
* SplitPart
* StringSplitSQL
Examples of incorrect results (under {{UTF8_LCASE}} collation):
{code:java}
SplitPart("Ai\u0307B", "İ", 2) // returns: "\u0307B" (incorrect), instead of:
"B" (correct)
SplitPart("AİB", "i\u0307", 1) // returns: "AİB", instead of: "A", "B"
(correct){code}
{code:java}
StringSplitSQL("Ai\u0307B", "İ") // returns: ["A", "\u0307B"] (incorrect),
instead of: ["A", "B"] (correct)
StringSplitSQL("AİB", "i\u0307") // returns: ["AİB"] (incorrect), instead of:
["A", "B"] (correct){code}
> Handle surrogate pairs properly
> -------------------------------
>
> Key: SPARK-49204
> URL: https://issues.apache.org/jira/browse/SPARK-49204
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Uroš Bojanić
> Priority: Major
> Labels: pull-request-available
>
> Fix the following string expressions to handle one-to-many case mapping
> properly:
> * SplitPart
> * StringSplitSQL
>
> Examples of incorrect results (under {{UTF8_LCASE}} collation):
> {code:java}
> SplitPart("Ai\u0307B", "İ", 2) // returns: "\u0307B" (incorrect), instead of:
> "B" (correct)
> SplitPart("AİB", "i\u0307", 1) // returns: "AİB", instead of: "A", "B"
> (correct){code}
>
> {code:java}
> StringSplitSQL("Ai\u0307B", "İ") // returns: ["A", "\u0307B"] (incorrect),
> instead of: ["A", "B"] (correct)
> StringSplitSQL("AİB", "i\u0307") // returns: ["AİB"] (incorrect), instead of:
> ["A", "B"] (correct){code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
