This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 5bee0db062d2 [SPARK-57250][SQL] Construct sub-microsecond timestamp
typed literals with precision derived from fractional digits
5bee0db062d2 is described below
commit 5bee0db062d251cd859136e3b2028cc9587dbf36
Author: Maxim Gekk <[email protected]>
AuthorDate: Fri Jun 5 07:44:40 2026 +0200
[SPARK-57250][SQL] Construct sub-microsecond timestamp typed literals with
precision derived from fractional digits
### What changes were proposed in this pull request?
Per the ANSI SQL standard (ISO/IEC 9075-2, Subclause 5.3, Syntax Rule 27),
the fractional-seconds precision of a typed timestamp literal is the number of
digits in its `<seconds fraction>`. This PR makes the typed timestamp literals
`TIMESTAMP '...'`, `TIMESTAMP_NTZ '...'`, and `TIMESTAMP_LTZ '...'` construct
nanosecond-capable values, deriving the precision `p` from the fractional-digit
count, when the SQL config `spark.sql.timestampNanosTypes.enabled` is enabled:
- 7-9 fractional digits -> `TimestampNTZNanosType(p)` /
`TimestampLTZNanosType(p)`
- `<=` 6 fractional digits -> existing microsecond behavior (unchanged)
- `>` 9 fractional digits -> new `INVALID_TIMESTAMP_LITERAL_PRECISION`
parse error
Concretely:
- Add `SparkDateTimeUtils.fractionalSecondsDigits(String): Int` to count
the digits of the seconds fraction in a timestamp string.
- Route `AstBuilder.visitTypeConstructor` for
`TIMESTAMP`/`TIMESTAMP_NTZ`/`TIMESTAMP_LTZ` to the nanosecond parse helpers
when the preview flag is on and the literal carries 7-9 fractional digits,
preserving the existing NTZ/LTZ resolution rules for the bare `TIMESTAMP`
keyword.
- Add the `INVALID_TIMESTAMP_LITERAL_PRECISION` error condition and a
`QueryParsingErrors.timestampLiteralPrecisionExceedsMaxError` factory for
literals with more than 9 fractional-second digits.
### Why are the changes needed?
To support standard-compliant sub-microsecond (nanosecond) timestamp typed
literals as part of the nanosecond timestamp preview umbrella (SPARK-56822).
The ANSI SQL literal grammar carries no explicit precision argument; the
precision is implied by the number of fractional-second digits, so the parser
must derive it from the literal text.
### Does this PR introduce _any_ user-facing change?
Yes, but only when the preview flag `spark.sql.timestampNanosTypes.enabled`
is set to `true`. In that case a typed timestamp literal with 7-9
fractional-second digits now produces a nanosecond-precision value instead of
being truncated to microseconds, and a literal with more than 9 fractional
digits raises `INVALID_TIMESTAMP_LITERAL_PRECISION`. With the flag off (the
default), behavior is unchanged: literals are parsed at microsecond precision
as before.
### How was this patch tested?
Added cases to `ExpressionParserSuite` ("SPARK-57250: nanosecond timestamp
typed literals") covering:
- 7/8/9-digit fractional precision for `TIMESTAMP_NTZ`, `TIMESTAMP_LTZ`,
and bare `TIMESTAMP`;
- NTZ/LTZ resolution based on the keyword and the presence of a time-zone
part;
- boundary values (pre-epoch, max year, `nanosWithinMicro` 0 and 999);
- the 6-digit-stays-microsecond regression;
- the `>9` fractional-digit error;
- the flag-off regression (microsecond behavior preserved).
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor (Claude Opus 4.8)
Closes #56306 from MaxGekk/spark-57250-typed-literals.
Authored-by: Maxim Gekk <[email protected]>
Signed-off-by: Uros Bojanic <[email protected]>
(cherry picked from commit 9e32a26d3782de4ccd4870d89164514db6e91e64)
Signed-off-by: Uros Bojanic <[email protected]>
---
.../src/main/resources/error/error-conditions.json | 6 ++
.../sql/catalyst/util/SparkDateTimeUtils.scala | 29 +++++++
.../spark/sql/errors/QueryParsingErrors.scala | 9 +++
.../spark/sql/catalyst/parser/AstBuilder.scala | 55 +++++++++++--
.../catalyst/parser/ExpressionParserSuite.scala | 94 +++++++++++++++++++++-
.../sql/catalyst/util/DateTimeUtilsSuite.scala | 29 +++++++
6 files changed, 213 insertions(+), 9 deletions(-)
diff --git a/common/utils/src/main/resources/error/error-conditions.json
b/common/utils/src/main/resources/error/error-conditions.json
index d4439631d8d0..a21f9aa08521 100644
--- a/common/utils/src/main/resources/error/error-conditions.json
+++ b/common/utils/src/main/resources/error/error-conditions.json
@@ -4944,6 +4944,12 @@
],
"sqlState" : "42K0F"
},
+ "INVALID_TIMESTAMP_LITERAL_PRECISION" : {
+ "message" : [
+ "The timestamp literal <value> has more than 9 fractional-second digits.
The maximum supported fractional-second precision of a timestamp literal is 9
(nanoseconds)."
+ ],
+ "sqlState" : "22023"
+ },
"INVALID_TIMESTAMP_PRECISION" : {
"message" : [
"The seconds precision <precision> of <type> is invalid. Expected an
integer in [7, 9] for nanosecond precision; use precision 6 or parameterless
<type> for the standard microsecond type."
diff --git
a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
index 5b08c965c055..f570d47ec8a0 100644
---
a/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
+++
b/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala
@@ -550,6 +550,35 @@ trait SparkDateTimeUtils {
}
}
+ /**
+ * Returns the number of fractional-second digits in a timestamp/time
string, i.e. the count of
+ * decimal digits immediately following the first `.` (0 if there is no
fractional part). In a
+ * well-formed timestamp/time string the only `.` is the one that introduces
the seconds
+ * fraction, so this is sufficient to derive the precision `p` of a typed
literal per the ANSI
+ * SQL rule (the precision of a timestamp literal is the number of digits in
its
+ * `<seconds fraction>`). Digits beyond the fractional run (e.g. a trailing
time zone) are not
+ * counted.
+ *
+ * This is intentionally a lightweight pre-parse digit counter: it does not
validate that `s` is
+ * a well-formed timestamp. Callers use the returned count only to choose a
parse path (the
+ * digit count routes between the microsecond path, the nanosecond path, and
the ">9 digits"
+ * error); each of those paths then re-parses and validates the whole
string, so a malformed
+ * input such as `"abcd.1234"` is still rejected downstream by the chosen
parser. Consequently
+ * the result is meaningful only for strings that are otherwise valid
timestamp/time literals.
+ */
+ def fractionalSecondsDigits(s: String): Int = {
+ val dot = s.indexOf('.')
+ if (dot < 0) {
+ 0
+ } else {
+ var i = dot + 1
+ while (i < s.length && s.charAt(i) >= '0' && s.charAt(i) <= '9') {
+ i += 1
+ }
+ i - (dot + 1)
+ }
+ }
+
/**
* Trims and parses a given UTF8 timestamp string to the corresponding
timestamp segments, time
* zone id and whether it is just time without a date. value. The return
type is [[Option]] in
diff --git
a/sql/api/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
b/sql/api/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
index 4879b8ba081f..1cc050f488f6 100644
---
a/sql/api/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
+++
b/sql/api/src/main/scala/org/apache/spark/sql/errors/QueryParsingErrors.scala
@@ -360,6 +360,15 @@ private[sql] object QueryParsingErrors extends
DataTypeErrorsBase {
ctx)
}
+ def timestampLiteralPrecisionExceedsMaxError(
+ value: String,
+ ctx: TypeConstructorContext): Throwable = {
+ new ParseException(
+ errorClass = "INVALID_TIMESTAMP_LITERAL_PRECISION",
+ messageParameters = Map("value" -> toSQLValue(value)),
+ ctx)
+ }
+
def literalValueTypeUnsupportedError(
unsupportedType: String,
supportedTypes: Seq[String],
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
index 66185933081a..1e33212ba12d 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala
@@ -46,7 +46,7 @@ import org.apache.spark.sql.catalyst.trees.{CurrentOrigin,
Origin}
import org.apache.spark.sql.catalyst.trees.TreePattern.PARAMETER
import org.apache.spark.sql.catalyst.types.DataTypeUtils
import org.apache.spark.sql.catalyst.util.{CharVarcharUtils, CollationFactory,
DateTimeUtils, EvaluateUnresolvedInlineTable, IntervalUtils}
-import org.apache.spark.sql.catalyst.util.DateTimeUtils.{convertSpecialDate,
convertSpecialTimestamp, convertSpecialTimestampNTZ, getZoneId, stringToDate,
stringToTime, stringToTimestamp, stringToTimestampWithoutTimeZone}
+import org.apache.spark.sql.catalyst.util.DateTimeUtils.{convertSpecialDate,
convertSpecialTimestamp, convertSpecialTimestampNTZ, fractionalSecondsDigits,
getZoneId, stringToDate, stringToTime, stringToTimestamp,
stringToTimestampLTZNanos, stringToTimestampNTZNanos,
stringToTimestampWithoutTimeZone}
import org.apache.spark.sql.connector.catalog.{CatalogV2Util,
ChangelogContext, PathElement, SupportsNamespaces, TableCatalog,
TableWritePrivilege}
import org.apache.spark.sql.connector.catalog.ChangelogRange.{TimestampRange,
UnboundedRange, VersionRange}
import org.apache.spark.sql.connector.catalog.TableChange.ColumnPosition
@@ -4048,6 +4048,39 @@ class AstBuilder extends DataTypeAstBuilder
specialTs.getOrElse(toLiteral(stringToTimestamp(_, zoneId),
TimestampType))
}
+ // ANSI SQL (ISO/IEC 9075-2, Subclause 5.3, Syntax Rule 27): the
fractional-seconds precision
+ // of a typed timestamp literal is the number of digits in its `<seconds
fraction>`. When the
+ // nanosecond preview is enabled and the literal carries 7-9 fractional
digits, build a
+ // nanosecond-capable literal with precision `p` equal to that digit
count. Literals with <= 6
+ // fractional digits keep the microsecond behavior; more than 9 digits is
rejected.
+ def constructTimestampNTZNanosLiteral(p: Int): Literal =
+ toLiteral(stringToTimestampNTZNanos(_, p), TimestampNTZNanosType(p))
+
+ def constructTimestampLTZNanosLiteral(p: Int): Literal = {
+ val zoneId = getZoneId(conf.sessionLocalTimeZone)
+ toLiteral(stringToTimestampLTZNanos(_, p, zoneId),
TimestampLTZNanosType(p))
+ }
+
+ // Returns Some(literal) when the nanos preview flag is on and the literal
has 7-9 fractional
+ // digits; throws when there are more than 9; returns None (fall back to
the micro path) when
+ // the flag is off or there are <= 6 fractional digits.
+ def nanosLiteralOpt(construct: Int => Literal): Option[Literal] = {
+ if (!SQLConf.get.timestampNanosTypesEnabled) {
+ None
+ } else {
+ val p = fractionalSecondsDigits(value)
+ // With the flag off, >9 fractional digits silently truncate to
microseconds via
+ // the fall-through path. Strict validation is intentionally
flag-gated.
+ if (p > TimestampNTZNanosType.MAX_PRECISION) {
+ throw
QueryParsingErrors.timestampLiteralPrecisionExceedsMaxError(value, ctx)
+ } else if (p >= TimestampNTZNanosType.MIN_PRECISION) {
+ Some(construct(p))
+ } else {
+ None
+ }
+ }
+ }
+
valueType match {
case DATE =>
val zoneId = getZoneId(conf.sessionLocalTimeZone)
@@ -4055,11 +4088,14 @@ class AstBuilder extends DataTypeAstBuilder
specialDate.getOrElse(toLiteral(stringToDate, DateType))
case TIME => toLiteral(stringToTime, TimeType())
case TIMESTAMP_NTZ =>
- convertSpecialTimestampNTZ(value, getZoneId(conf.sessionLocalTimeZone))
- .map(Literal(_, TimestampNTZType))
- .getOrElse(toLiteral(stringToTimestampWithoutTimeZone,
TimestampNTZType))
+ nanosLiteralOpt(constructTimestampNTZNanosLiteral).getOrElse {
+ convertSpecialTimestampNTZ(value,
getZoneId(conf.sessionLocalTimeZone))
+ .map(Literal(_, TimestampNTZType))
+ .getOrElse(toLiteral(stringToTimestampWithoutTimeZone,
TimestampNTZType))
+ }
case TIMESTAMP_LTZ =>
- constructTimestampLTZLiteral(value)
+ nanosLiteralOpt(constructTimestampLTZNanosLiteral)
+ .getOrElse(constructTimestampLTZLiteral(value))
case TIMESTAMP =>
SQLConf.get.timestampType match {
case TimestampNTZType =>
@@ -4071,14 +4107,17 @@ class AstBuilder extends DataTypeAstBuilder
// If the input string contains time zone part, return a
timestamp with local time
// zone literal.
if (containsTimeZonePart) {
- constructTimestampLTZLiteral(value)
+ nanosLiteralOpt(constructTimestampLTZNanosLiteral)
+ .getOrElse(constructTimestampLTZLiteral(value))
} else {
- toLiteral(stringToTimestampWithoutTimeZone, TimestampNTZType)
+ nanosLiteralOpt(constructTimestampNTZNanosLiteral)
+ .getOrElse(toLiteral(stringToTimestampWithoutTimeZone,
TimestampNTZType))
}
}
case TimestampType =>
- constructTimestampLTZLiteral(value)
+ nanosLiteralOpt(constructTimestampLTZNanosLiteral)
+ .getOrElse(constructTimestampLTZLiteral(value))
}
case INTERVAL =>
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
index 08d64d2db1c5..758e7eaf2158 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala
@@ -17,7 +17,7 @@
package org.apache.spark.sql.catalyst.parser
import java.sql.{Date, Timestamp}
-import java.time.{Duration, LocalDateTime, LocalTime, Period}
+import java.time.{Duration, LocalDateTime, LocalTime, Period, ZoneOffset}
import java.util.concurrent.TimeUnit
import scala.language.implicitConversions
@@ -1184,6 +1184,98 @@ class ExpressionParserSuite extends AnalysisTest {
}
}
+ test("SPARK-57250: nanosecond timestamp typed literals") {
+ import org.apache.spark.sql.catalyst.util.TimestampNanosTestUtils._
+
+ // Expected NTZ / LTZ nanos literals from readable components. The session
time zone is fixed
+ // to UTC below so the wall-clock fields of the LTZ literal map to the
same instant.
+ def ntz(p: Int, y: Int, mo: Int, d: Int, h: Int, mi: Int, s: Int,
nanoOfSec: Int): Literal =
+ Literal(localDateTimeToNanosVal(timestampNTZ(y, mo, d, h, mi, s,
nanoOfSec)),
+ TimestampNTZNanosType(p))
+ def ltz(p: Int, y: Int, mo: Int, d: Int, h: Int, mi: Int, s: Int,
nanoOfSec: Int): Literal =
+ Literal(instantToNanosVal(timestampLTZ(y, mo, d, h, mi, s, nanoOfSec)),
+ TimestampLTZNanosType(p))
+
+ withSQLConf(
+ SQLConf.TIMESTAMP_NANOS_TYPES_ENABLED.key -> "true",
+ SQLConf.SESSION_LOCAL_TIMEZONE.key -> "UTC") {
+ // Precision is derived from the number of fractional digits (ANSI SQL
Subclause 5.3 SR 27).
+ assertEqual("TIMESTAMP_NTZ '2020-01-01 00:00:00.1234567'",
+ ntz(7, 2020, 1, 1, 0, 0, 0, 123456700))
+ assertEqual("TIMESTAMP_NTZ '2020-01-01 00:00:00.12345678'",
+ ntz(8, 2020, 1, 1, 0, 0, 0, 123456780))
+ assertEqual("TIMESTAMP_NTZ '2020-01-01 00:00:00.123456789'",
+ ntz(9, 2020, 1, 1, 0, 0, 0, 123456789))
+
+ // TIMESTAMP_LTZ: value interpreted in the session time zone (UTC here).
+ assertEqual("TIMESTAMP_LTZ '2020-01-01 00:00:00.123456789'",
+ ltz(9, 2020, 1, 1, 0, 0, 0, 123456789))
+
+ // TIMESTAMP_LTZ with an explicit zone offset in the literal: the offset
takes precedence
+ // over the session timezone. '2020-01-01 00:00:00.123456789+05:00' is
the instant
+ // 2019-12-31 19:00:00.123456789 UTC.
+ assertEqual("TIMESTAMP_LTZ '2020-01-01 00:00:00.123456789+05:00'",
+ Literal(
+ instantToNanosVal(timestampLTZ(2020, 1, 1, 0, 0, 0, 123456789,
ZoneOffset.of("+05:00"))),
+ TimestampLTZNanosType(9)))
+
+ // Bare TIMESTAMP keyword resolves to LTZ nanos by default
(TIMESTAMP_TYPE = LTZ).
+ assertEqual("TIMESTAMP '2020-01-01 00:00:00.123456789'",
+ ltz(9, 2020, 1, 1, 0, 0, 0, 123456789))
+
+ // Under the NTZ default, bare TIMESTAMP resolves to NTZ nanos, unless
the string carries a
+ // time-zone offset, which flips it to LTZ nanos.
+ withSQLConf(SQLConf.TIMESTAMP_TYPE.key ->
TimestampTypes.TIMESTAMP_NTZ.toString) {
+ assertEqual("TIMESTAMP '2020-01-01 00:00:00.123456789'",
+ ntz(9, 2020, 1, 1, 0, 0, 0, 123456789))
+ assertEqual("TIMESTAMP '2020-01-01 00:00:00.123456789+00:00'",
+ ltz(9, 2020, 1, 1, 0, 0, 0, 123456789))
+ }
+
+ // Boundary values: nanosWithinMicro 0 and 999; pre-epoch (1582) and the
max year (9999).
+ assertEqual("TIMESTAMP_NTZ '1970-01-01 00:00:00.000000000'",
+ ntz(9, 1970, 1, 1, 0, 0, 0, 0))
+ assertEqual("TIMESTAMP_NTZ '1970-01-01 00:00:00.000000999'",
+ ntz(9, 1970, 1, 1, 0, 0, 0, 999))
+ assertEqual("TIMESTAMP_NTZ '1582-10-15 23:59:59.123456789'",
+ ntz(9, 1582, 10, 15, 23, 59, 59, 123456789))
+ assertEqual("TIMESTAMP_NTZ '9999-12-31 23:59:59.999999999'",
+ ntz(9, 9999, 12, 31, 23, 59, 59, 999999999))
+
+ // Exactly 6 fractional digits stays a microsecond literal.
+ assertEqual("TIMESTAMP_NTZ '2020-01-01 00:00:00.123456'",
+ Literal(LocalDateTime.parse("2020-01-01T00:00:00.123456")))
+
+ // More than 9 fractional digits is rejected.
+ checkError(
+ exception = parseException("TIMESTAMP_NTZ '2020-01-01
00:00:00.1234567890'"),
+ condition = "INVALID_TIMESTAMP_LITERAL_PRECISION",
+ parameters = Map("value" -> "'2020-01-01 00:00:00.1234567890'"),
+ context = ExpectedContext(
+ fragment = "TIMESTAMP_NTZ '2020-01-01 00:00:00.1234567890'",
+ start = 0,
+ stop = 45))
+
+ // Special values have no fractional part, so nanosLiteralOpt returns
None and the
+ // existing special-value path handles them, producing plain microsecond
literals.
+ assertEqual("TIMESTAMP_NTZ 'epoch'", Literal(0L, TimestampNTZType))
+ }
+
+ // With the preview flag off, 7-9 digit literals narrow to microseconds
(legacy behavior).
+ withSQLConf(
+ SQLConf.TIMESTAMP_NANOS_TYPES_ENABLED.key -> "false",
+ SQLConf.SESSION_LOCAL_TIMEZONE.key -> "UTC") {
+ assertEqual("TIMESTAMP_NTZ '2020-01-01 00:00:00.123456789'",
+ Literal(LocalDateTime.parse("2020-01-01T00:00:00.123456")))
+
+ // More than 9 fractional digits is NOT rejected when the flag is off;
the strict
+ // INVALID_TIMESTAMP_LITERAL_PRECISION validation is intentionally
flag-gated, so the
+ // literal silently narrows to microseconds via the legacy fall-through
path.
+ assertEqual("TIMESTAMP_NTZ '2020-01-01 00:00:00.1234567890'",
+ Literal(LocalDateTime.parse("2020-01-01T00:00:00.123456")))
+ }
+ }
+
test("date literals") {
DateTimeTestUtils.outstandingTimezonesIds.foreach { timeZone =>
withSQLConf(SQLConf.SESSION_LOCAL_TIMEZONE.key -> timeZone) {
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
index 2d9793e687a7..8256da9fb3f9 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala
@@ -412,6 +412,35 @@ class DateTimeUtilsSuite extends SparkFunSuite with
Matchers with SQLHelper {
None)
}
+ test("SPARK-57250: fractionalSecondsDigits counts digits after the first
dot") {
+ // No fractional part.
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00") === 0)
+ assert(fractionalSecondsDigits("2020-01-01") === 0)
+ assert(fractionalSecondsDigits("") === 0)
+
+ // A trailing dot with no digits.
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.") === 0)
+
+ // Boundary digit counts used by the literal precision routing.
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.1") === 1)
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.123456") === 6)
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.1234567") === 7)
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.123456789") === 9)
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.1234567890") === 10)
+
+ // Counting stops at the first non-digit, e.g. a trailing time zone or
whitespace.
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.123456789+08:00") ===
9)
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.123 ") === 3)
+ assert(fractionalSecondsDigits("12:30:45.5Z") === 1)
+
+ // Only the first dot introduces the fraction; later dots are not counted.
+ assert(fractionalSecondsDigits("2020-01-01 00:00:00.12.34") === 2)
+
+ // The helper does not validate the rest of the string; it just counts the
fractional run.
+ assert(fractionalSecondsDigits("abcd.1234") === 4)
+ assert(fractionalSecondsDigits(".789") === 3)
+ }
+
test("SPARK-15379: special invalid date string") {
// Test stringToDate
assert(toDate("2015-02-29 00:00:00").isEmpty)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
