[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592: URL: https://github.com/apache/spark/pull/28592#discussion_r429117869 ## File path: docs/sql-ref-datetime-pattern.md ## @@ -26,44 +26,128 @@ There are several common scenarios for datetime usage in Spark: - Datetime functions related to convert `StringType` to/from `DateType` or `TimestampType`. For example, `unix_timestamp`, `date_format`, `to_unix_timestamp`, `from_unixtime`, `to_date`, `to_timestamp`, `from_utc_timestamp`, `to_utc_timestamp`, etc. -Spark uses pattern letters in the following table for date and timestamp parsing and formatting: - -|Symbol|Meaning|Presentation|Examples| -|--|---||| -|**G**|era|text|AD; Anno Domini; A| -|**y**|year|year|2020; 20| -|**D**|day-of-year|number|189| -|**M/L**|month-of-year|number/text|7; 07; Jul; July; J| -|**d**|day-of-month|number|28| -|**Q/q**|quarter-of-year|number/text|3; 03; Q3; 3rd quarter| -|**Y**|week-based-year|year|1996; 96| -|**w**|week-of-week-based-year|number|27| -|**W**|week-of-month|number|4| -|**E**|day-of-week|text|Tue; Tuesday; T| -|**u**|localized day-of-week|number/text|2; 02; Tue; Tuesday; T| -|**F**|week-of-month|number|3| -|**a**|am-pm-of-day|text|PM| -|**h**|clock-hour-of-am-pm (1-12)|number|12| -|**K**|hour-of-am-pm (0-11)|number|0| -|**k**|clock-hour-of-day (1-24)|number|0| -|**H**|hour-of-day (0-23)|number|0| -|**m**|minute-of-hour|number|30| -|**s**|second-of-minute|number|55| -|**S**|fraction-of-second|fraction|978| -|**V**|time-zone ID|zone-id|America/Los_Angeles; Z; -08:30| -|**z**|time-zone name|zone-name|Pacific Standard Time; PST| -|**O**|localized zone-offset|offset-O|GMT+8; GMT+08:00; UTC-08:00;| -|**X**|zone-offset 'Z' for zero|offset-X|Z; -08; -0830; -08:30; -083015; -08:30:15;| -|**x**|zone-offset|offset-x|+; -08; -0830; -08:30; -083015; -08:30:15;| -|**Z**|zone-offset|offset-Z|+; -0800; -08:00;| -|**'**|escape for text|delimiter| | -|**''**|single quote|literal|'| -|**[**|optional section start| | | -|**]**|optional section end| | | - -The count of pattern letters determines the format. - -- Text: The text style is determined based on the number of pattern letters used. Less than 4 pattern letters will use the short form. Exactly 4 pattern letters will use the full form. Exactly 5 pattern letters will use the narrow form. Six or more letters will fail. +The following tables define how the pattern letters be used for date and timestamp parsing and formatting in Spark. + +## Date Fields + +Pattern letters to output a date: + +|Pattern|Count|Meaning|Presentation|Examples| +|---|---|---|---|---| +|**G**|1|era|text|AD| +|**GG**|2|era|text|AD| +|**GGG**|3|era|text|AD| Review comment: I tried this once, but some of the patterns are not able to be simplified because of different outputs. So I choose to stick to same layout. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r429110475
##
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##
@@ -750,3 +750,164 @@ select to_timestamp("2019 10:10:10", " hh:mm:ss")
struct
-- !query output
2019-01-01 10:10:10
+
+
+-- !query
+select date_format(date '2020-05-23', 'G')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'G' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(date '2020-05-23', 'M')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'M' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(date '2020-05-23', 'L')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'L' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(timestamp '2020-05-23', 'E')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'E' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(timestamp '2020-05-23', 'u')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'u' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format('2020-05-23', 'Q')
+-- !query schema
+struct<>
+-- !query output
+java.lang.IllegalArgumentException
+Too many pattern letters: Q
+
+
+-- !query
+select date_format('2020-05-23', 'q')
+-- !query schema
+struct<>
+-- !query output
+java.lang.IllegalArgumentException
+Too many pattern letters: q
+
+
+-- !query
+select to_timestamp('2019-10-06 A', '-MM-dd G')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize '-MM-dd G' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select to_timestamp('22 05 2020 Friday', 'dd MM EE')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'dd MM EE' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select to_timestamp('22 05 2020 Friday', 'dd MM E')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'dd MM E' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select unix_timestamp('22 05 2020 Friday', 'dd MM E')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r429110475
##
File path: sql/core/src/test/resources/sql-tests/results/datetime.sql.out
##
@@ -750,3 +750,164 @@ select to_timestamp("2019 10:10:10", " hh:mm:ss")
struct
-- !query output
2019-01-01 10:10:10
+
+
+-- !query
+select date_format(date '2020-05-23', 'G')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'G' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(date '2020-05-23', 'M')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'M' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(date '2020-05-23', 'L')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'L' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(timestamp '2020-05-23', 'E')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'E' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format(timestamp '2020-05-23', 'u')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'u' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select date_format('2020-05-23', 'Q')
+-- !query schema
+struct<>
+-- !query output
+java.lang.IllegalArgumentException
+Too many pattern letters: Q
+
+
+-- !query
+select date_format('2020-05-23', 'q')
+-- !query schema
+struct<>
+-- !query output
+java.lang.IllegalArgumentException
+Too many pattern letters: q
+
+
+-- !query
+select to_timestamp('2019-10-06 A', '-MM-dd G')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize '-MM-dd G' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select to_timestamp('22 05 2020 Friday', 'dd MM EE')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'dd MM EE' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select to_timestamp('22 05 2020 Friday', 'dd MM E')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the upgrading of Spark 3.0: Fail to
recognize 'dd MM E' pattern in the new parser. 1) You can set
spark.sql.legacy.timeParserPolicy to LEGACY to restore the behavior before
Spark 3.0. 2) You can form a valid datetime pattern with the guide from
https://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html
+
+
+-- !query
+select unix_timestamp('22 05 2020 Friday', 'dd MM E')
+-- !query schema
+struct<>
+-- !query output
+org.apache.spark.SparkUpgradeException
+You may get a different result due to the
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r429093201
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala
##
@@ -33,6 +34,8 @@ sealed trait DateFormatter extends Serializable {
def format(days: Int): String
def format(date: Date): String
def format(localDate: LocalDate): String
+
+ def initialize(): Unit = {}
Review comment:
Got it. SGTM.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428667113
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala
##
@@ -40,7 +43,23 @@ class Iso8601DateFormatter(
extends DateFormatter with DateTimeFormatterHelper {
@transient
- private lazy val formatter = getOrCreateFormatter(pattern, locale)
+ private lazy val formatter = {
Review comment:
Hmm, this one and the others are transient, so the lazy keyword is
required.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428667113
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala
##
@@ -40,7 +43,23 @@ class Iso8601DateFormatter(
extends DateFormatter with DateTimeFormatterHelper {
@transient
- private lazy val formatter = getOrCreateFormatter(pattern, locale)
+ private lazy val formatter = {
Review comment:
Hmm, this one and the other are transient, so the lazy keyword is
required.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428607225
##
File path: sql/core/src/test/resources/sql-tests/inputs/datetime-corrected.sql
##
@@ -0,0 +1,2 @@
+--SET spark.sql.legacy.timeParserPolicy=CORRECTED
Review comment:
I've just come to understand what it means, I will rm this case.
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TimestampFormatter.scala
##
@@ -61,8 +64,23 @@ class Iso8601TimestampFormatter(
needVarLengthSecondFraction: Boolean)
extends TimestampFormatter with DateTimeFormatterHelper {
@transient
- protected lazy val formatter: DateTimeFormatter =
-getOrCreateFormatter(pattern, locale, needVarLengthSecondFraction)
+ protected lazy val formatter: DateTimeFormatter = {
+try {
Review comment:
OK
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428606030
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala
##
@@ -40,7 +43,23 @@ class Iso8601DateFormatter(
extends DateFormatter with DateTimeFormatterHelper {
@transient
- private lazy val formatter = getOrCreateFormatter(pattern, locale)
+ private lazy val formatter = {
Review comment:
Shall we remove this lazy to let it fail fast in the parse phase?
@cloud-fan
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428604890
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala
##
@@ -77,14 +96,14 @@ trait LegacyDateFormatter extends DateFormatter {
class LegacyFastDateFormatter(pattern: String, locale: Locale) extends
LegacyDateFormatter {
@transient
- private lazy val fdf = FastDateFormat.getInstance(pattern, locale)
Review comment:
needed
https://github.com/apache/spark/pull/28592/files#diff-7f589e01d3e5e5ea284c1622527d4984R52
here, to fail if invaild
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428000557
##
File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql
##
@@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12",
"-MM-dd'T'HH:mm:ss''"); --
select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail
select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head
select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); --
head but as single quote
+
+-- Unsupported narrow text style
+select date_format(date '2020-05-23', 'G');
Review comment:
For parsing and formatting, throw SparkUpgradeExeception to tell users
how to restore the previous behavior or update their pattern via our online
doc? How about the current approach
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r428000557
##
File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql
##
@@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12",
"-MM-dd'T'HH:mm:ss''"); --
select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail
select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head
select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); --
head but as single quote
+
+-- Unsupported narrow text style
+select date_format(date '2020-05-23', 'G');
Review comment:
For parsing and formatting, throw SparkUpgradeExeception to tell users
how to restore or update their pattern via our online doc? How about the
current approach
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r427960736
##
File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql
##
@@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12",
"-MM-dd'T'HH:mm:ss''"); --
select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail
select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head
select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); --
head but as single quote
+
+-- Unsupported narrow text style
+select date_format(date '2020-05-23', 'G');
Review comment:
I just realize that returning NULL is also silent data change...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] yaooqinn commented on a change in pull request #28592: [SPARK-31771][SQL] Disable Narrow TextStyle for datetime pattern 'G/M/L/E/u/Q/q'
yaooqinn commented on a change in pull request #28592:
URL: https://github.com/apache/spark/pull/28592#discussion_r427956371
##
File path: sql/core/src/test/resources/sql-tests/inputs/datetime.sql
##
@@ -122,3 +122,13 @@ select to_timestamp("2019-10-06T10:11:12'12",
"-MM-dd'T'HH:mm:ss''"); --
select to_timestamp("2019-10-06T10:11:12'", "-MM-dd'T'HH:mm:ss''"); -- tail
select to_timestamp("'2019-10-06T10:11:12", "''-MM-dd'T'HH:mm:ss"); -- head
select to_timestamp("P2019-10-06T10:11:12", "'P'-MM-dd'T'HH:mm:ss"); --
head but as single quote
+
+-- Unsupported narrow text style
+select date_format(date '2020-05-23', 'G');
Review comment:
This question may beyond the scope of this PR. So far, using an invalid
pattern for parsing always returns `NULL`, which is also compatible with 2.4.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
