cloud-fan commented on a change in pull request #32959:
URL: https://github.com/apache/spark/pull/32959#discussion_r664235831
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##########
@@ -224,12 +224,12 @@ object DateTimeUtils {
* value. The return type is [[Option]] in order to distinguish between 0L
and null. The following
* formats are allowed:
*
- * `yyyy`
- * `yyyy-[m]m`
- * `yyyy-[m]m-[d]d`
- * `yyyy-[m]m-[d]d `
- * `yyyy-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
- * `yyyy-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
+ * `[+-]y*`
+ * `[+-]y*-[m]m`
+ * `[+-]y*-[m]m-[d]d`
+ * `[+-]y*-[m]m-[d]d `
+ * `[+-]y*-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
+ * `[+-]y*-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
* `[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
Review comment:
are you sure? It seems we only consider the sign for year:
https://github.com/apache/spark/pull/32959/files#diff-f59286f6576476c68ae766de0430fc41d424804dbb0658e0dccd998b0d9a756dR404
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##########
@@ -368,12 +410,12 @@ object DateTimeUtils {
* value. The return type is [[Option]] in order to distinguish between 0L
and null. The following
* formats are allowed:
*
- * `yyyy`
Review comment:
Maybe we should just say "Please refer to parseTimestampString for the
allowed formats." here
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##########
@@ -425,12 +467,12 @@ object DateTimeUtils {
* The return type is [[Option]] in order to distinguish between 0L and
null. The following
* formats are allowed:
*
- * `yyyy`
- * `yyyy-[m]m`
- * `yyyy-[m]m-[d]d`
- * `yyyy-[m]m-[d]d `
- * `yyyy-[m]m-[d]d [h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
- * `yyyy-[m]m-[d]dT[h]h:[m]m:[s]s.[ms][ms][ms][us][us][us][zone_id]`
+ * `[+-]y*`
Review comment:
ditto
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala
##########
@@ -518,44 +560,56 @@ object DateTimeUtils {
* The return type is [[Option]] in order to distinguish between 0 and null.
The following
* formats are allowed:
*
- * `yyyy`
- * `yyyy-[m]m`
- * `yyyy-[m]m-[d]d`
- * `yyyy-[m]m-[d]d `
- * `yyyy-[m]m-[d]d *`
- * `yyyy-[m]m-[d]dT*`
+ * `[+-]y*`
Review comment:
ditto
##########
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/AnsiCastSuiteBase.scala
##########
@@ -392,8 +391,6 @@ abstract class AnsiCastSuiteBase extends CastSuiteBase {
s"Cannot cast $str to DateType.")
}
- checkCastWithParseError("12345")
- checkCastWithParseError("12345-12-18")
Review comment:
let's move the removed tests to the base cast suite
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
