gengliangwang commented on a change in pull request #33115:
URL: https://github.com/apache/spark/pull/33115#discussion_r659841530
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
##########
@@ -217,3 +217,14 @@ private[sql] abstract class FractionalType extends
NumericType {
private[sql] val fractional: Fractional[InternalType]
private[sql] val asIntegral: Integral[InternalType]
}
+
+private[sql] object AnyTimestampType extends AbstractDataType with
Serializable {
Review comment:
I do have introduced `TypeCollection.AllTimestampTypes` in a previous PR.
However, in this PR I find that having `AnyTimestampType` can simplify the
code changes in `TypeCoercion.scala`.
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
##########
@@ -217,3 +217,14 @@ private[sql] abstract class FractionalType extends
NumericType {
private[sql] val fractional: Fractional[InternalType]
private[sql] val asIntegral: Integral[InternalType]
}
+
+private[sql] object AnyTimestampType extends AbstractDataType with
Serializable {
Review comment:
Yes, I will have a follow-up for that.
There is another reason, in the default type coercion rules, when implicit
casting a type to a `TypeCollection` type, Spark chooses the first convertible
data type as the result. If we are going to make the default timestamp type
configurable, having `AnyTimestampType` is better.
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
##########
@@ -217,3 +217,14 @@ private[sql] abstract class FractionalType extends
NumericType {
private[sql] val fractional: Fractional[InternalType]
private[sql] val asIntegral: Integral[InternalType]
}
+
+private[sql] object AnyTimestampType extends AbstractDataType with
Serializable {
Review comment:
So we should have such a new type for types of the same category, like
`NumericType` and `IntegralType`.
If the data types are of different category, it's fine to use
`TypeCollection`
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala
##########
@@ -217,3 +217,14 @@ private[sql] abstract class FractionalType extends
NumericType {
private[sql] val fractional: Fractional[InternalType]
private[sql] val asIntegral: Integral[InternalType]
}
+
+private[sql] object AnyTimestampType extends AbstractDataType with
Serializable {
Review comment:
So we should have such a new type for types of the same category, like
`NumericType` and `IntegralType`.
If the data types are of different categories, it's fine to use
`TypeCollection`
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala
##########
@@ -386,8 +386,8 @@ class Analyzer(override val catalogManager: CatalogManager)
DatetimeSub(l, r, DateAddInterval(l, UnaryMinus(r, f), ansiEnabled
= f))
case (_, CalendarIntervalType | _: DayTimeIntervalType) =>
Cast(DatetimeSub(l, r, TimeAdd(l, UnaryMinus(r, f))), l.dataType)
- case (TimestampType, _) => SubtractTimestamps(l, r)
- case (_, TimestampType) => SubtractTimestamps(l, r)
+ case (TimestampType | TimestampWithoutTZType, _) =>
SubtractTimestamps(l, r)
Review comment:
No. But we can simplify it with
```
case _ if AnyTimestampType.unapply(l) || AnyTimestampType.unapply(r) =>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
