Github user HyukjinKwon commented on a diff in the pull request:
https://github.com/apache/spark/pull/14829#discussion_r76409496
--- Diff:
sql/hive/src/test/scala/org/apache/spark/sql/sources/JsonHadoopFsRelationSuite.scala
---
@@ -32,10 +32,6 @@ class JsonHadoopFsRelationSuite extends
HadoopFsRelationTest {
override protected def supportsDataType(dataType: DataType): Boolean =
dataType match {
case _: NullType => false
case _: BinaryType => false
- // `TimestampType` is disabled because
`DatatypeConverter.parseDateTime()`
- // in `DateTimeUtils` parses the formatted string wrongly when the
date is
- // too early. (e.g. "1600-07-13T08:36:32.847").
- case _: TimestampType => false
--- End diff --
It should have been enabled in the previous PR. My initial proposal was
just to change the write path where both CSV and JSON used we used
`DatatypeConverter.parseDateTime` for read path.
However, the previous PR ended up with changing read path as well switching
`DatatypeConverter.parseDateTime` to `FastDateFormat` not having this problem
as described above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
