LuciferYang opened a new pull request #34237:
URL: https://github.com/apache/spark/pull/34237
### What changes were proposed in this pull request?
The `date_format` function with `B` format has different behavior when use
Java 8 and Java 17, `select date_format('2018-11-17 13:33:33.333', 'B')` in
`datetime-formatting-invalid.sql` can prove this.
The case result with Java 8 is
```
-- !query
select date_format('2018-11-17 13:33:33.333', 'B')
-- !query schema
struct<>
-- !query output
java.lang.IllegalArgumentException
Unknown pattern letter: B
```
and the case result with Java 17 is
```
- datetime-formatting-invalid.sql *** FAILED ***
datetime-formatting-invalid.sql
Expected "struct<[]>", but got "struct<[date_format(2018-11-17
13:33:33.333, B):string]>" Schema did not match for query #34
select date_format('2018-11-17 13:33:33.333', 'B'): -- !query
select date_format('2018-11-17 13:33:33.333', 'B')
-- !query schema
struct<date_format(2018-11-17 13:33:33.333, B):string>
-- !query output
in the afternoon (SQLQueryTestSuite.scala:469)
```
We found that this is due to the new support of format `B` in Java 17
```
'B' is used to represent Pattern letters to output a day period in Java 17
* Pattern Count Equivalent builder methods
* ------- ----- --------------------------
* B 1 appendDayPeriodText(TextStyle.SHORT)
* BBBB 4 appendDayPeriodText(TextStyle.FULL)
* BBBBB 5 appendDayPeriodText(TextStyle.NARROW)
```
And through [
http://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html](
http://spark.apache.org/docs/latest/sql-ref-datetime-pattern.html) , we can
confirm that format `B` is not documented/supported for `date_format` function
currently.
So the main change of this pr is manual disabled format `B` of `date_format`
function in `DateTimeFormatterHelper` to make Java 17 compatible with Java 8.
### Why are the changes needed?
Ensure that Java 17 and Java 8 have the same behavior.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Pass the Jenkins or GitHub Action
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
