[
https://issues.apache.org/jira/browse/SPARK-47210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihailo Milosevic updated SPARK-47210:
--------------------------------------
Parent: SPARK-47624
Issue Type: Sub-task (was: Improvement)
> Implicit casting on collated expressions
> ----------------------------------------
>
> Key: SPARK-47210
> URL: https://issues.apache.org/jira/browse/SPARK-47210
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Mihailo Milosevic
> Priority: Major
> Labels: pull-request-available
>
> *What changes were proposed in this pull request?*
> This PR adds automatic casting and collations resolution as per `PGSQL`
> behaviour:
> 1. Collations set on the metadata level are implicit
> 2. Collations set using the `COLLATE` expression are explicit
> 3. When there is a combination of expressions of multiple collations the
> output will be:
> - if there are explicit collations and all of them are equal then that
> collation will be the output
> - if there are multiple different explicit collations
> `COLLATION_MISMATCH.EXPLICIT` will be thrown
> - if there are no explicit collations and only a single type of non default
> collation, that one will be used
> - if there are no explicit collations and multiple non-default implicit ones
> `COLLATION_MISMATCH.IMPLICIT` will be thrown
> Another thing is that `INDETERMINATE_COLLATION` should only be thrown on
> comparison operations, and we should be able to combine different implicit
> collations for certain operations like concat and possible others in the
> future.
> This is why I had to add another predefined collation id named
> `INDETERMINATE_COLLATION_ID` which means that the result is a combination of
> conflicting non-default implicit collations. Right now it has an id of -1 so
> it fails if it ever goes to the `CollatorFactory`.
> *Why are the changes needed?*
> We need to be able to compare columns and values with different collations
> and set a way of explicitly changing the collation we want to use.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
