[
https://issues.apache.org/jira/browse/SPARK-52351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17955276#comment-17955276
]
Mithun Radhakrishnan commented on SPARK-52351:
----------------------------------------------
Question to the Spark devs: Is this the new expected behaviour for Apache
Spark, when parsing timestamps with leading whitespaces?
I ask because https://github.com/apache/spark/pull/44463 is labeled as having
"no user-facing changes". But this seems to be a change in semantics for the
user.
> Casting string to timestamp can not handle pattern: spaces + Thh:mm:ss
> ----------------------------------------------------------------------
>
> Key: SPARK-52351
> URL: https://issues.apache.org/jira/browse/SPARK-52351
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: chong
> Priority: Major
>
> There is a difference between Spark350 and Spark400 when handing pattern:
> space followed by a {{T}}
> h3. Spark350:
> {{scala> spark.sql("select cast(' T23:17:50' as timestamp)").show()}}
> {{2025-05-30 23:17:50}}
> h3. Spark400:
> {{spark.conf.set("spark.sql.ansi.enabled", false)}}
> {{spark.sql("select cast(' T23:17:50' as timestamp)").show()}}
> {{ NULL}}
> {{}}
> h3. bug explain
> [Code
> link|https://github.com/apache/spark/blob/v4.0.0/sql/api/src/main/scala/org/apache/spark/sql/catalyst/util/SparkDateTimeUtils.scala#L450-L468]
> var j = getTrimmedStart(bytes) // ============== here j maybe
> not zero after trim leftval strEndTrimmed = getTrimmedEnd(j, bytes) if (j
> == strEndTrimmed) { return (Array.empty, None, false)
> } var digitsMilli = 0var justTime = falsevar yearSign: Option[Int] =
> Noneif (bytes(j) == '-' || bytes(j) == '+') {
> yearSign = if (bytes(j) == '-') Some(-1) else Some(1)
> j += 1 } while (j < strEndTrimmed) { val b = bytes(j)
> val parsedValue = b - '0'.toByte if (parsedValue < 0 || parsedValue > 9)
> { if (j == 0 && b == 'T') {
> justTime = true // ============== never reach to here if trimed
> left and pattern is Thh:mm:ss i += 3 } else if (i < 2) {{{}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
