mihailom-db commented on code in PR #48585:
URL: https://github.com/apache/spark/pull/48585#discussion_r1810840747
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala:
##########
@@ -148,6 +149,10 @@ object AnsiTypeCoercion extends TypeCoercionBase {
// interval type the string should be promoted as. There are many
possible interval
// types, such as year interval, month interval, day interval, hour
interval, etc.
case (_: StringType, _: AnsiIntervalType) => None
+ // [SPARK-50060] If a binary operation contains at least one collated
string types, we can't
+ // decide which collation the result should have.
+ case (d1: StringType, d2: StringType) if
SchemaUtils.hasNonUTF8BinaryCollation(d1) ||
Review Comment:
By design we should not hit this rule. The CollationTypeCast is done first
in the TypeCoercion and AnsiTypeCoercion rules. @cloud-fan am I missing
something here?
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CollationTypeCasts.scala:
##########
@@ -200,4 +199,15 @@ object CollationTypeCasts extends TypeCoercionRule {
}
}
}
+
+ /**
+ * Returns whether the expression has explicit collated output type.
+ */
+ def hasExplicitCollation(expr: Expression): Boolean = {
+ expr match {
+ case _: Collate => true
+ case alias: Alias => hasExplicitCollation(alias.child)
Review Comment:
Are we sure this is safe to do for all other expressions not related to your
PR? It might be that some expressions gets a subquery as input, could you
please make sure we do not break something with propagating explicit collation
like this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
