imback82 commented on pull request #29328: URL: https://github.com/apache/spark/pull/29328#issuecomment-673180206
> LGTM except one question: do we still need this fix? https://github.com/apache/spark/pull/29328/files#r468330253 Yes, without it, the following tests fail: ``` org.apache.spark.sql.execution.datasources.csv.CSVLegacyTimeParserSuite.SPARK-23846: schema inferring touches less data if samplingRatio < 1.0 org.apache.spark.sql.execution.datasources.csv.CSVv1Suite.SPARK-23846: schema inferring touches less data if samplingRatio < 1.0 org.apache.spark.sql.execution.datasources.csv.CSVv2Suite.SPARK-23846: schema inferring touches less data if samplingRatio < 1.0 org.apache.spark.sql.execution.datasources.json.JsonLegacyTimeParserSuite.SPARK-11544 test pathfilter org.apache.spark.sql.execution.datasources.json.JsonLegacyTimeParserSuite.SPARK-23849: schema inferring touches less data if samplingRatio < 1.0 org.apache.spark.sql.execution.datasources.json.JsonV1Suite.SPARK-11544 test pathfilter org.apache.spark.sql.execution.datasources.json.JsonV1Suite.SPARK-23849: schema inferring touches less data if samplingRatio < 1.0 org.apache.spark.sql.execution.datasources.json.JsonV2Suite.SPARK-11544 test pathfilter org.apache.spark.sql.execution.datasources.json.JsonV2Suite.SPARK-23849: schema inferring touches less data if samplingRatio < 1.0 ``` ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
