Aleksandar Tomic created SPARK-46837:
----------------------------------------
Summary: String function support (parent)
Key: SPARK-46837
URL: https://issues.apache.org/jira/browse/SPARK-46837
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 4.0.0
Reporter: Aleksandar Tomic
TODO: List of all functions that need to be updated for collation support:
|Feature/function|Priority|Type|
|Shuffle|0|comparison|
|Delta Columns|0|storage|
|Partition key|0|storage|
|Comparison operators|0|comparison|
|IN list|0|comparison|
|GROUP BY |0|comparison|
|MERGE, HASH joins|0|comparison|
|ORDER BY |0|sorting|
|Aggregation|0|comparison|
|like|0|comparison|
|regexp_*|0|matching|
|concat|0|Pass through|
|substr|0|Pass through|
|between|0|comparison|
|coalesce |0|Pass through|
|Is distinct|0|comparison|
|*trim*|0|Pass through|
|instr|0|comparison|
|lcase|0|Pass through, modify|
|lower|0|Pass through modify|
|replace|0|comparison|
|ucase|0|modify , pass through|
|upper|0|Modify, pass through|
|count(distinct )|0|comparison|
|min/max|0|Comparison pass through|
|array|0|Pass through|
|case|0|Pass through|
|decode|0|Pass through|
|elt|0|Comparison, pass through|
|Nullif, nvl, nvl2|0|Pass through, comparison|
| | | |
|Session variables|1|storage|
|SQL UDF|1|Storage, pass through|
|Python UDF|1|Storage|
|Array element|1|storage|
|Map key|1|Storage, comparison|
|Map value|1|Storage|
|Struct field|1|storage|
|least/greatest|1|Comparison, pass through|
|if/iff/ifnull|1|Pass through, comparison|
|mapExpr [ keyExpr ]|1|Comparison, pass through|
|concat_ws|1|Pass through|
|contains|1|comparison|
|left|1|Pass through|
|*pad|1|Pass through|
|repeat|1|Pass through|
|reverse|1|Pass through|
|translate|1|Comparison, Pass through|
|array_agg|1|Pass through|
|first/last/any|1|Pass through|
|mode|1|Comparison, pass through|
|array_*|1|Pass through, dedup (array distinct)|
| | | |
|*explode*|2|Pass through|
|filter|2|Pass through|
|flatten|2|Pass through|
|inline*|2|Pass through|
|reduce|2|Pass through|
|reverse|2|Pass through|
|shuffle|2|Pass through|
|Slice|2|Pass through|
|sort_array|2|Comparison, pass through|
|transform|2|Pass through|
|zip*|2|Pass through|
|map|2|Pass through|
|map_*|2|Pass through|
|str_to_map|2|Comparison, pass through|
|transform*|2|Pass through|
|stack|2|Pass through|
|describe|2|display|
|ilike|2|matching|
|charindex|2|comparison|
|endswith|2|comparison|
|startswith|2|comparison|
|find_in_set|2|comparison|
|initcap|2|Pass through, modify|
|locate|2|comparison|
|mask|2|Pass through|
|overlay|2|Pass through|
|position|2|comparison|
|sentences|2|Comparison, pass through|
|split|2|Comparison, pass through|
|split_part|2|Comparison, pass through|
|collect_list|2|Pass through|
|collect_set|2|Pass through|
|min_by/max_by|2|Comparison, pass through|
|Element_at, []|2|Pass through|
|aggregate|2|Pass through|
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
