LuciferYang commented on PR #36774: URL: https://github.com/apache/spark/pull/36774#issuecomment-1147244030
https://github.com/apache/spark/blob/cc0bf563b8caea21da5692f05e34b5f77e002ab9/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala#L146-L161 After SPARK-37463: For `OrcFileFormat`, `org.apache.orc.FileFormatException` is change to thrown in line 146(no longer line 156) and is always handled by `FileScanRDD`, whether `ignoreCorruptFiles` is true or not. A similar situation exists for `OrcPartitionReaderFactory`, so should we change to use `OrcUtils.readSchema(filePath, conf, ignoreCorruptFiles)` instead? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
