MaxGekk commented on a change in pull request #23391: [SPARK-26456][SQL] Cast
date/timestamp to string by Date/TimestampFormatter
URL: https://github.com/apache/spark/pull/23391#discussion_r244422244
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateFormatter.scala
##########
@@ -88,11 +88,18 @@ class LegacyFallbackDateFormatter(
}
object DateFormatter {
+ val defaultPattern: String = "yyyy-MM-dd"
+ val defaultLocale: Locale = Locale.US
+
def apply(format: String, locale: Locale): DateFormatter = {
if (SQLConf.get.legacyTimeParserEnabled) {
Review comment:
In this PR, date and timestamp patterns are fixed, and we shouldn't see any
behavior changes but `DateFormatter`/`TimestampFormatter` are used from
CSV/JSON Datasources and from a functions where user can set any patterns.
Unfortunately supported patterns by `SimpleDateFormat` and `DateTimeFormat` are
not absolutely the same. Also there are other differences in their behavior:
https://github.com/apache/spark/pull/23358/files#diff-3f19ec3d15dcd8cd42bb25dde1c5c1a9R42
What I have learned from other PRs, if I introduce a behavior change, I
should leave opportunity to users to come back to previous behavior. Later the
old behavior can be deprecated.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
