subhramit commented on code in PR #58225:
URL: https://github.com/apache/spark/pull/58225#discussion_r3950122457
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/FileSourceCustomMetadataStructSuite.scala:
##########
@@ -413,6 +413,21 @@ class FileSourceCustomMetadataStructSuite extends
SharedSparkSession {
Row(1, 112L, 1L, f1.getLen, f1.getPath.getName)))
}
}
+
+ test("SPARK-58945: invalid file metadata fields report the field") {
+ withTempData("parquet", FILE_SCHEMA) { (_, f0, f1) =>
+ val invalidField = StructField("bad", StringType)
+ val format = new TestFileFormat(Seq(invalidField))
+ val df = createDF(format, Seq(FileStatusWithMetadata(f0),
FileStatusWithMetadata(f1)))
+
+ checkError(
+ exception = intercept[AnalysisException] {
+ df.select("_metadata.bad").collect()
+ },
+ condition = "_LEGACY_ERROR_TEMP_3070",
+ parameters = Map("field" -> invalidField.toString))
Review Comment:
Also on the original `loadTable(...)` test yes that path does not reach
`H2Dialect.classifyException`. I updated the test to cover the reachable direct
`renameTable(...)` path instead, then fixed the branch to fall back to
`oldName` when `tableName` is absent. So I’ll edit the PR description as this
is not demonstrated by `loadTable(...)`, but it is no longer just a
hypothetical path
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
