fanyue-xia commented on code in PR #49590:
URL: https://github.com/apache/spark/pull/49590#discussion_r1926008820
##########
connector/avro/src/test/scala/org/apache/spark/sql/avro/AvroCatalystDataConversionSuite.scala:
##########
@@ -241,6 +241,25 @@ class AvroCatalystDataConversionSuite extends SparkFunSuite
}
}
+ test("schema nullability mismatch") {
+ val innerSchema = StructType(Seq(StructField("inner", StringType, false)))
+ val nestedSchema = StructType(Seq(StructField("outer", innerSchema,
false)))
+
+ val nestedRow = Row.fromSeq(Seq(null))
+ val converter =
CatalystTypeConverters.createToCatalystConverter(nestedSchema)
+ val input = Literal.create(converter(nestedRow), nestedSchema)
+ checkError(
+ exception = intercept[SparkRuntimeException] {
+ CatalystDataToAvro(input, None).eval()
Review Comment:
Thanks for the feedback. Reproduced.
Updated the code to check nullability according to `avroType` instead, since
it is used during `write`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
