MaxGekk commented on code in PR #45723:
URL: https://github.com/apache/spark/pull/45723#discussion_r1540537958
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala:
##########
@@ -805,33 +799,34 @@ private[sql] object QueryExecutionErrors extends
QueryErrorsBase with ExecutionE
cause = null)
}
- def fileNotFoundError(e: FileNotFoundException): SparkFileNotFoundException
= {
- new SparkFileNotFoundException(
- errorClass = "_LEGACY_ERROR_TEMP_2062",
- messageParameters = Map("message" -> e.getMessage))
+ def fileNotExistError(path: String, e: Exception): Throwable = {
+ new SparkException(
Review Comment:
Why did you change `SparkFileNotFoundException` to `SparkException`. This
can break user's apps, I think. If they catch the exact
`SparkFileNotFoundException`, they will miss your `SparkException`. Especially,
after you changed the error class too. How should user know about your changes?
##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -1257,6 +1249,31 @@
"message" : [
"Encountered error while reading file <path>."
],
+ "subClass" : {
+ "CANNOT_READ_FILE_FOOTER" : {
+ "message" : [
+ "Could not read footer. Please ensure that the file is in either ORC
or Parquet format.",
+ "If not, please convert it to a valid format. If the file is in the
valid format, please check if it is corrupt.",
+ "If it is, you can choose to either ignore it or fix the corruption."
+ ]
+ },
+ "FILE_NOT_EXIST" : {
+ "message" : [
+ "File does not exist. It is possible the underlying files have been
updated.",
+ "You can explicitly invalidate the cache in Spark by running
'REFRESH TABLE tableName' command in SQL or by recreating the Dataset/DataFrame
involved."
+ ]
+ },
+ "NO_HINT" : {
+ "message" : [
+ ""
+ ]
+ },
+ "PARQUET_COLUMN_DATA_TYPE_MISMATCH" : {
+ "message" : [
+ "Data type mismatches when reading Parquet column <column>:
Expected: <expectedType>, Found: <actualType>."
Review Comment:
too much `:`.
```suggestion
"Data type mismatches when reading Parquet column <column>.
Expected <expectedType> but found <actualType>."
```
##########
common/utils/src/main/resources/error/error-classes.json:
##########
@@ -1257,6 +1249,31 @@
"message" : [
"Encountered error while reading file <path>."
],
+ "subClass" : {
+ "CANNOT_READ_FILE_FOOTER" : {
+ "message" : [
+ "Could not read footer. Please ensure that the file is in either ORC
or Parquet format.",
+ "If not, please convert it to a valid format. If the file is in the
valid format, please check if it is corrupt.",
+ "If it is, you can choose to either ignore it or fix the corruption."
+ ]
+ },
+ "FILE_NOT_EXIST" : {
+ "message" : [
+ "File does not exist. It is possible the underlying files have been
updated.",
+ "You can explicitly invalidate the cache in Spark by running
'REFRESH TABLE tableName' command in SQL or by recreating the Dataset/DataFrame
involved."
+ ]
+ },
+ "NO_HINT" : {
+ "message" : [
+ ""
+ ]
+ },
+ "PARQUET_COLUMN_DATA_TYPE_MISMATCH" : {
+ "message" : [
+ "Data type mismatches when reading Parquet column <column>:
Expected: <expectedType>, Found: <actualType>."
Review Comment:
BTW, since we are talking about the parquet format, shouldn't we mention
that one type is a logical type, but another is a physical type? Otherwise you
compare apples to oranges.
##########
connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroLogicalTypeSuite.scala:
##########
@@ -436,7 +436,7 @@ abstract class AvroLogicalTypeSuite extends QueryTest with
SharedSparkSession {
val ex = intercept[SparkException] {
spark.read.format("avro").load(s"$dir.avro").collect()
}
- assert(ex.getErrorClass == "FAILED_READ_FILE")
+ assert(ex.getErrorClass.startsWith("FAILED_READ_FILE"))
Review Comment:
Why don't compare to `FAILED_READ_FILE.NO_HINT`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
