linhongliu-db commented on a change in pull request #32959:
URL: https://github.com/apache/spark/pull/32959#discussion_r662803317
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
##########
@@ -2684,16 +2682,13 @@ abstract class JsonSuite
}
test("SPARK-30960, SPARK-31641: parse date/timestamp string with legacy
format") {
- val julianDay = -141704 // 1582-01-01 in Julian calendar
val ds = Seq(
- s"{'t': '2020-1-12 3:23:34.12', 'd': '2020-1-12 T', 'd2': '12345', 'd3':
'$julianDay'}"
Review comment:
'12345' and '-141704' are treated as epoch days before this PR because
it's out of the 0000-9999 range.
this is used for backward compatibility with JSON data generated by spark
1.5.
But this compatibility is very confusing, for example, before this PR:
'9999' will be converted to '9999-01-01' while '10000' will be converted to
'1997-05-19'
So I suggest just removing this compatibility
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
