SaurabhChawla100 commented on a change in pull request #32972:
URL: https://github.com/apache/spark/pull/32972#discussion_r656895066
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveUnion.scala
##########
@@ -182,6 +182,15 @@ object ResolveUnion extends Rule[LogicalPlan] {
// union have consistent schema.
aliased += foundAttr
Alias(addFields(foundAttr, target), foundAttr.name)()
+ case (source: StructType, target: StructType)
+ if !allowMissingCol && !source.sameType(target) &&
+ target.toAttributes.map(attr => attr.name).sorted
+ == source.toAttributes.map(x => x.name).sorted =>
Review comment:
This validation is not required
```target.toAttributes.map(attr => attr.name).sorted ==
source.toAttributes.map(x => x.name).sorted ```
Since the we need to sort both left and right side in case of struct and
sortStructFields method recursively sorts in all the nested struct.
Now after removing this validation case sensitive attributes with different
sequence are also working
as its working for the case sensitive attributes with same sequence
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
