cloud-fan commented on a change in pull request #23196: [SPARK-26243][SQL] Use java.time API for parsing timestamps and dates from JSON URL: https://github.com/apache/spark/pull/23196#discussion_r241468755
########## File path: sql/hive/src/test/scala/org/apache/spark/sql/sources/HadoopFsRelationTest.scala ########## @@ -125,56 +126,58 @@ abstract class HadoopFsRelationTest extends QueryTest with SQLTestUtils with Tes } else { Seq(false) } - for (dataType <- supportedDataTypes) { - for (parquetDictionaryEncodingEnabled <- parquetDictionaryEncodingEnabledConfs) { - val extraMessage = if (isParquetDataSource) { - s" with parquet.enable.dictionary = $parquetDictionaryEncodingEnabled" - } else { - "" - } - logInfo(s"Testing $dataType data type$extraMessage") - - val extraOptions = Map[String, String]( - "parquet.enable.dictionary" -> parquetDictionaryEncodingEnabled.toString - ) - - withTempPath { file => - val path = file.getCanonicalPath - - val dataGenerator = RandomDataGenerator.forType( - dataType = dataType, - nullable = true, - new Random(System.nanoTime()) - ).getOrElse { - fail(s"Failed to create data generator for schema $dataType") + withSQLConf(SQLConf.SESSION_LOCAL_TIMEZONE.key -> "UTC") { Review comment: > converting milliseconds to Java's Timestamp and to something else. these don't matter once the dataframe is created. The problem is, if we have a dataframe(no matter how it is generated), we write it out and read it back. If it becomes different, we have a bug. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org