caican00 opened a new pull request, #41993:
URL: https://github.com/apache/spark/pull/41993
### What changes were proposed in this pull request?
The input/output Attribute is preferred for validation using the
`__CHAR_VARCHAR_TYPE_STRING` type specified in its Metadata.
If not specified, use the Attribute datatype again.
### Why are the changes needed?
If the input/output Attribute specifies a `__CHAR_VARCHAR_TYPE_STRING` tag
in its Metadata, then the type specified by `__CHAR_VARCHAR_TYPE_STRING` should
be used for validation.
Otherwise, use the Attribute DataType for verification to avoid the
following exception：
```
org.apache.spark.sql.AnalysisException: unresolved operator 'AppendData
TestRelation [x#8, y#9], true;
'AppendData TestRelation [x#8, y#9], true
+- TestRelation [x#6, y#7]
```
How to reproduce？
```
val analyzer = getAnalyzer
// check varchar type
val json1 = "{\"__CHAR_VARCHAR_TYPE_STRING\":\"varchar(80)\"}"
val metadata1 = new
MetadataBuilder().withMetadata(Metadata.fromJson(json1)).build()
val query1 = TestRelation(StructType(Seq(
StructField("x", StringType, metadata = metadata1),
StructField("y", StringType, metadata = metadata1))).toAttributes)
val table1 = TestRelation(StructType(Seq(
StructField("x", StringType, metadata = metadata1),
StructField("y", StringType, metadata = metadata1))).toAttributes)
val parsedPlanByName1 = byName(table1, query1)
analyzer.executeAndCheck(parsedPlanByName1, new QueryPlanningTracker())
```
### Does this PR introduce _any_ user-facing change?
Yes. The above code works fine.
### How was this patch tested?
new UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
