This is an automated email from the ASF dual-hosted git repository.
wenchen pushed a commit to branch branch-3.0
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.0 by this push:
new 424f16e [SPARK-33015][SQL] Compute the current date only once
424f16e is described below
commit 424f16e8cbe7e9ba02e808bf084cd7361388e377
Author: Max Gekk <[email protected]>
AuthorDate: Tue Sep 29 05:13:01 2020 +0000
[SPARK-33015][SQL] Compute the current date only once
### What changes were proposed in this pull request?
Compute the current date at the specified time zone using timestamp taken
at the start of query evaluation.
### Why are the changes needed?
According to the doc for
[current_date()](http://spark.apache.org/docs/latest/api/sql/#current_date),
the current date should be computed at the start of query evaluation but it can
be computed multiple times. As a consequence of that, the function can return
different values if the query is executed at the border of two dates.
### Does this PR introduce _any_ user-facing change?
Yes
### How was this patch tested?
By existing test suites `ComputeCurrentTimeSuite` and
`DateExpressionsSuite`.
Closes #29889 from MaxGekk/fix-current_date.
Authored-by: Max Gekk <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 68cd5677ae0e3891e6bb4938a64ff98810656ba8)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
index 80d8582..09e0118 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/finishAnalysis.scala
@@ -79,10 +79,10 @@ object ComputeCurrentTime extends Rule[LogicalPlan] {
val currentTime = Literal.create(timestamp, timeExpr.dataType)
plan transformAllExpressions {
- case CurrentDate(Some(timeZoneId)) =>
+ case currentDate @ CurrentDate(Some(timeZoneId)) =>
currentDates.getOrElseUpdate(timeZoneId, {
Literal.create(
- LocalDate.now(DateTimeUtils.getZoneId(timeZoneId)),
+ DateTimeUtils.microsToDays(timestamp, currentDate.zoneId),
DateType)
})
case CurrentTimestamp() | Now() => currentTime
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
