yaooqinn commented on pull request #28692:
URL: https://github.com/apache/spark/pull/28692#issuecomment-637608841
> why was Monday the first day of week in 2.4 then?
In Spark version 2.4 and earlier, datetime parsing and formatting are
performed by the old Java 7 `SimpleDateFormat` API, the ` Monday the first day
of week ` thing is decided by SimpleDateFormat. Since Spark 3.0, we switch to
the new Java 8 `DateTimeFormatter` to use the Proleptic Gregorian calendar,
which is required by the ISO and SQL standards.
> we didn't use TZ to determine it?
The `first day of week` is a date field, not a time field, TZ is short for
time zone actually.
> Also is that the right answer from a SQL standard
The `first day of week` is not directly decided by ANSI SQL Framework but
from its normative reference ISO standard. It is Monday start in ISO. The day
of week`DateTimeFormatter` become localized and by default we use `Locale.US`
where a week starts from Sunday. We have to restore this for backward
compatibility. There may more solutions options provided at SPARK-31879
> or Hive perspective?
I think the backward compatibility between 3.0 and 2.4 shall go first.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
