yaooqinn opened a new pull request #28692:
URL: https://github.com/apache/spark/pull/28692
# What changes were proposed in this pull request?
This PR switches the default Locale from the `US` to `GB` to change the
behavior of the first day of the week from Sunday-started to Monday-started as
same as v2.4
### Why are the changes needed?
#### cases
{code:sql}
spark-sql> select to_timestamp('2020-1-1', 'YYYY-w-u');
2019-12-29 00:00:00
spark-sql> set spark.sql.legacy.timeParserPolicy=legacy;
spark.sql.legacy.timeParserPolicy legacy
spark-sql> select to_timestamp('2020-1-1', 'YYYY-w-u');
2019-12-30 00:00:00
{code}
#### reasons
These week-based fields need Locale to express their semantics, the first
day of the week varies from country to country.
From the Java doc of WeekFields
```java
/**
* Gets the first day-of-week.
* <p>
* The first day-of-week varies by culture.
* For example, the US uses Sunday, while France and the ISO-8601
standard use Monday.
* This method returns the first day using the standard {@code
DayOfWeek} enum.
*
* @return the first day-of-week, not null
*/
public DayOfWeek getFirstDayOfWeek() {
return firstDayOfWeek;
}
```
But for the SimpleDateFormat, the day-of-week is not localized
```
u Day number of week (1 = Monday, ..., 7 = Sunday) Number 1
```
Currently, the default locale we use is the US, so the result moved a day
backward.
For other countries, please refer to [First Day of the Week in Different
Countries|http://chartsbin.com/view/41671]
With this change, it restores the first day of week calculating for
functions when using the default locale.
### Does this PR introduce _any_ user-facing change?
Yes, but the behavior change is used to restore the old one of v2.4
### How was this patch tested?
add unit tests
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
