maropu commented on a change in pull request #26644: [SPARK-30004][SQL] Allow
merge UserDefinedType into a native DataType
URL: https://github.com/apache/spark/pull/26644#discussion_r350485657
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/UserDefinedTypeSuite.scala
##########
@@ -287,4 +293,63 @@ class UserDefinedTypeSuite extends QueryTest with
SharedSparkSession with Parque
checkAnswer(spark.createDataFrame(data, schema).selectExpr("typeof(a)"),
Seq(Row("array<double>")))
}
+
+ test("Allow merge UserDefinedType into a native DataType") {
Review comment:
How about the simplified one like this?
```
test("Allow merge UserDefinedType into a native DataType") {
// Register the UDT
UDTRegistration.register(
classOf[XMLGregorianCalendar].getName,
classOf[MyXMLGregorianCalendarUDT].getName)
withTempDir { dir =>
val gregorianCalendar = new GregorianCalendar(1925, 5, 20, 19, 25)
// Equivalent of above (the year minus 1900)
val timestamp = new Timestamp(25, 5, 20, 19, 25, 0, 0)
val data = Seq(Row(new XMLGregorianCalendarImpl(gregorianCalendar)))
val rdd = spark.sparkContext.parallelize(data)
val schema = StructType(StructField("dt", new
MyXMLGregorianCalendarUDT) :: Nil)
val df = spark.sqlContext.createDataFrame(rdd, schema)
df.write.mode(SaveMode.Append).save(dir.getCanonicalPath)
// We should be able to write a second time, and Spark should be able
to resolve the types
df.write.mode(SaveMode.Append).save(dir.getCanonicalPath)
val records = spark.read.load(dir.getCanonicalPath)
records.printSchema()
checkAnswer(records, Row(timestamp) :: Row(timestamp) :: Nil)
}
}
```
But, it seems this current test is not related to this fix (I mean this test
passed without the fix). Can you check again?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
