uros-b commented on code in PR #59243:
URL: https://github.com/apache/spark/pull/59243#discussion_r4294401743
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:
##########
@@ -3294,6 +3294,26 @@ case class ParseToTimestamp(
) +: format.map(_ => StringTypeWithCollation(supportsTrimCollation =
true)).toSeq
}
Review Comment:
In
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala:2046,
:2086, :3238 and
sql/api/src/main/scala/org/apache/spark/sql/functions.scala:12662 — These
input-type docs still say numeric input is accepted, without saying that this
only holds when `fmt` is omitted. That now contradicts the type error this PR
makes official.
- The three `datetimeExpressions.scala` lines are the `timestamp_str`
entries in the `arguments` blocks of `to_timestamp_ltz`, `try_to_timestamp` and
`to_timestamp`: "An expression that evaluates to a string, date, timestamp, or
numeric." Users see this text in the SQL function reference and in `DESCRIBE
FUNCTION EXTENDED`.
- `functions.scala:12662` is the `@param s` doc of `try_to_timestamp(s:
Column, format: Column)`. That overload always takes a format, so numeric input
is never valid for it.
- SPARK-58085 wrote these lines by hand, based on what each expression
accepts and rejects. They should get a qualifier, e.g. "a string, date, or
timestamp; or a numeric when `fmt` is omitted". The PySpark docstrings have the
same sentence (python/pyspark/sql/functions/builtin.py:12841, :12995, :14750)
and can be updated at the same time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
