Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/19664#discussion_r149633100
--- Diff:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/ScalaReflectionSuite.scala
---
@@ -335,4 +338,17 @@ class ScalaReflectionSuite extends SparkFunSuite {
assert(linkedHashMapDeserializer.dataType ==
ObjectType(classOf[LHMap[_, _]]))
}
+ test("SPARK-22442: Generate correct field names for special characters")
{
+ val serializer = serializerFor[SpecialCharAsFieldData](BoundReference(
+ 0, ObjectType(classOf[SpecialCharAsFieldData]), nullable = false))
+ val deserializer = deserializerFor[SpecialCharAsFieldData]
+ assert(serializer.dataType(0).name == "field.1")
+ assert(serializer.dataType(1).name == "field 2")
+
+ val argumentsFields =
deserializer.asInstanceOf[NewInstance].arguments.flatMap { _.collect {
+ case UpCast(u: UnresolvedAttribute, _, _) => u.name
--- End diff --
actually we should collect `u.nameParts` here. `u.name` adds backticks if
the `nameParts` contains dot, which makes the test result a little confusing.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
