HyukjinKwon commented on a change in pull request #25782: [SPARK-29074][SQL]
Optimize `date_format` for foldable `fmt`
URL: https://github.com/apache/spark/pull/25782#discussion_r324488922
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala
##########
@@ -580,7 +580,11 @@ case class WeekOfYear(child: Expression) extends
UnaryExpression with ImplicitCa
""",
since = "1.5.0")
// scalastyle:on line.size.limit
-case class DateFormatClass(left: Expression, right: Expression, timeZoneId:
Option[String] = None)
+case class DateFormatClass(
+ left: Expression,
+ right: Expression,
+ timeZoneId: Option[String] = None,
+ formatter: Option[TimestampFormatter] = None)
extends BinaryExpression with TimeZoneAwareExpression with
ImplicitCastInputTypes {
Review comment:
@MaxGekk, BTW, why don't we use lazy val? For instance,
```scala
@transient private lazy val formatter: Option[TimestampFormatter] = {
if (right.foldable && timeZoneId.isDefined) {
val format = right.eval().toString
Some(TimestampFormatter(format, getZoneId(timeZoneId.get)))
} else {
None
}
}
```
I suspect you combined this as `TimestampFormatter` is dependent on timezone
and `TimeZoneAwareExpression` rule. But that's in analyzer which must happen
before actual execution so I think this way could work.
I just didn't like combining `withTimeZone` with another caching logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
