This is an automated email from the ASF dual-hosted git repository.
MaxGekk pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 70b343025a5a [SPARK-57588][SQL] Report the actual target precision in
CAST_INVALID_INPUT when casting strings to TIME(p)
70b343025a5a is described below
commit 70b343025a5a917760dd383c6d589cfe571c0f02
Author: Maxim Gekk <[email protected]>
AuthorDate: Mon Jun 22 21:24:27 2026 +0200
[SPARK-57588][SQL] Report the actual target precision in CAST_INVALID_INPUT
when casting strings to TIME(p)
### What changes were proposed in this pull request?
When a string cannot be parsed while casting to `TIME(p)` in ANSI mode, the
`CAST_INVALID_INPUT` error always reported the target type as `"TIME(6)"`,
regardless of the actual target precision. This happened because
`SparkDateTimeUtils.stringToTimeAnsi` hardcoded the default `TimeType()`
(precision 6) when constructing the error, and the cast's target precision
never reached the error builder.
This PR:
- Adds a precision-aware overload `stringToTimeAnsi(s, precision, context)`
in `SparkDateTimeUtils` that reports `TimeType(precision)` in the error. The
existing no-arg/default overload is kept (delegating to
`TimeType.DEFAULT_PRECISION`) for non-cast callers such as
`DefaultTimeFormatter.parse`.
- Updates `Cast.castToTime` (interpreted) and `Cast.castToTimeCode`
(codegen) to pass the cast's target `to.precision`.
This mirrors the existing nanosecond `TIMESTAMP` casts, which already pass
the real target type, e.g. `invalidInputInCastToDatetimeError(s,
TimestampLTZNanosType(precision), ...)`.
### Why are the changes needed?
The `CAST_INVALID_INPUT` error message was inaccurate for any TIME
precision other than 6. For example:
```sql
SET spark.sql.ansi.enabled=true;
SELECT CAST('not a time' AS TIME(2));
```
reported `targetType = "TIME(6)"` instead of the requested `"TIME(2)"`.
### Does this PR introduce _any_ user-facing change?
Yes. With ANSI mode enabled, `CAST('x' AS TIME(p))` on invalid input now
reports `targetType = "TIME(p)"` (the requested precision) instead of always
`"TIME(6)"`. This is a change only within the unreleased master/branch lines.
### How was this patch tested?
- Extended `DateTimeUtilsSuite."stringToTimeAnsi"` to assert the error
reports the requested precision across the supported range
`[TimeType.MIN_PRECISION, TimeType.MAX_PRECISION]`, in addition to the existing
default `TIME(6)` cases.
- Added `CastWithAnsiOnSuite."SPARK-57588: cast invalid string input to
time reports the target precision"`, which exercises both the interpreted and
codegen paths via the standard cast eval harness.
Ran `build/sbt "catalyst/testOnly *DateTimeUtilsSuite
*CastWithAnsiOnSuite"` (199 tests passed) and scalastyle on `sql-api` and
`catalyst` (0 errors).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor (Claude Opus 4.8)
Closes #56641 from MaxGekk/time-fix-CAST_INVALID_INPUT.
Authored-by: Maxim Gekk <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
(cherry picked from commit ecaa9cf9509d25e1866ca0461842391e654efa89)
Signed-off-by: Max Gekk <[email protected]>
---
.../spark/sql/catalyst/util/SparkDateTimeUtils.scala | 7 +++++--
.../apache/spark/sql/catalyst/expressions/Cast.scala | 5 +++--
.../sql/catalyst/expressions/CastWithAnsiOnSuite.scala | 9 +++++++++
.../spark/sql/catalyst/util/DateTimeUtilsSuite.scala | 17 ++++++++++++++++-
4 files changed, 33 insertions(+), 5 deletions(-)
diff --git
a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
index 430b04d8f3df..5abc80e7c6a1 100644
---
a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
+++
b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
@@ -1123,9 +1123,12 @@ trait SparkDateTimeUtils {
}
}
- def stringToTimeAnsi(s: UTF8String, context: QueryContext = null): Long = {
+ def stringToTimeAnsi(s: UTF8String, context: QueryContext = null): Long =
+ stringToTimeAnsi(s, TimeType.DEFAULT_PRECISION, context)
+
+ def stringToTimeAnsi(s: UTF8String, precision: Int, context: QueryContext):
Long = {
stringToTime(s).getOrElse {
- throw ExecutionErrors.invalidInputInCastToDatetimeError(s, TimeType(),
context)
+ throw ExecutionErrors.invalidInputInCastToDatetimeError(s,
TimeType(precision), context)
}
}
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
index 62bbd65863e8..d12f46ecee9f 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala
@@ -966,7 +966,7 @@ case class Cast(
case _: StringType =>
if (ansiEnabled) {
buildCast[UTF8String](_, s => DateTimeUtils.truncateTimeToPrecision(
- DateTimeUtils.stringToTimeAnsi(s, getContextOrNull()), to.precision))
+ DateTimeUtils.stringToTimeAnsi(s, to.precision, getContextOrNull()),
to.precision))
} else {
buildCast[UTF8String](_, s => DateTimeUtils.stringToTime(s)
.map(DateTimeUtils.truncateTimeToPrecision(_, to.precision)).orNull)
@@ -1673,7 +1673,8 @@ case class Cast(
val errorContext = getContextOrNullCode(ctx)
code"""
$evPrim = $dateTimeUtilsCls.truncateTimeToPrecision(
- $dateTimeUtilsCls.stringToTimeAnsi($c, $errorContext),
${to.precision});
+ $dateTimeUtilsCls.stringToTimeAnsi($c, ${to.precision},
$errorContext),
+ ${to.precision});
"""
} else {
code"""
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastWithAnsiOnSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastWithAnsiOnSuite.scala
index ce7850d8c9c1..ef3e1e6eca9b 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastWithAnsiOnSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastWithAnsiOnSuite.scala
@@ -868,6 +868,15 @@ class CastWithAnsiOnSuite extends CastSuiteBase with
QueryErrorsBase {
}
}
+ test("SPARK-57588: cast invalid string input to time reports the target
precision") {
+ for (precision <- TimeType.MIN_PRECISION to TimeType.MAX_PRECISION) {
+ val invalidInput = "not a time"
+ checkExceptionInExpression[DateTimeException](
+ cast(invalidInput, TimeType(precision)),
+ castErrMsg(invalidInput, TimeType(precision)))
+ }
+ }
+
test("SPARK-52620: cast time to decimal with insufficient precision (ANSI
on)") {
// Create a time that will overflow Decimal(2, 0): 23:59:59 = 86399
seconds.
val largeTime = Literal.create(LocalTime.of(23, 59, 59, 123456000),
TimeType(6))
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
index 0a6c9123a20e..1f594667fce3 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
@@ -34,8 +34,8 @@ import org.apache.spark.sql.catalyst.util.DateTimeUtils._
import
org.apache.spark.sql.catalyst.util.RebaseDateTime.rebaseJulianToGregorianMicros
import org.apache.spark.sql.errors.DataTypeErrors.toSQLConf
import org.apache.spark.sql.internal.SqlApiConf
+import org.apache.spark.sql.types.{Decimal, TimeType}
import org.apache.spark.sql.types.DayTimeIntervalType.{HOUR, SECOND}
-import org.apache.spark.sql.types.Decimal
import org.apache.spark.unsafe.types.{CalendarInterval, TimestampNanosVal,
UTF8String}
class DateTimeUtilsSuite extends SparkFunSuite with Matchers with SQLHelper {
@@ -1505,6 +1505,21 @@ class DateTimeUtilsSuite extends SparkFunSuite with
Matchers with SQLHelper {
"targetType" -> "\"TIME(6)\"",
"ansiConfig" -> "\"spark.sql.ansi.enabled\""))
}
+
+ // The error must report the requested target precision, not the default
TIME(6).
+ for (precision <- TimeType.MIN_PRECISION to TimeType.MAX_PRECISION) {
+ val invalidTime = "not a time"
+ checkError(
+ exception = intercept[SparkDateTimeException] {
+ stringToTimeAnsi(UTF8String.fromString(invalidTime), precision, null)
+ },
+ condition = "CAST_INVALID_INPUT",
+ parameters = Map(
+ "expression" -> s"'$invalidTime'",
+ "sourceType" -> "\"STRING\"",
+ "targetType" -> s""""TIME($precision)"""",
+ "ansiConfig" -> "\"spark.sql.ansi.enabled\""))
+ }
}
test("timeToMicros") {
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
