maropu commented on a change in pull request #27926: [SPARK-31166][SQL] UNION
map<null, null> and other maps should not fail
URL: https://github.com/apache/spark/pull/27926#discussion_r393389988
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala
##########
@@ -3487,6 +3487,12 @@ class SQLQuerySuite extends QueryTest with
SharedSparkSession with AdaptiveSpark
}
}
}
+
+ test("SPARK-31166: UNION map<null, null> and other maps should not fail") {
+ checkAnswer(
+ sql("(SELECT map()) UNION ALL (SELECT map(1, 2))"),
+ Seq(Row(Map[Int, Int]()), Row(Map(1 -> 2))))
+ }
Review comment:
Nice catch! It seems #27542 broke some other queries merging
`map<null,null>` and `map<some type, some type>`, e.g., `map_concat`;
```
// the current master
scala> sql("select map_concat(map(), map(1, 2))").show()
org.apache.spark.sql.AnalysisException: cannot resolve 'map_concat(map(),
map(1, 2))' due to data type mismatch: input to function map_concat should all
be the same type, but it's [map<null,null>, map<int,int>]; line 1 pos 7;
'Project [unresolvedalias(map_concat(map(), map(1, 2)), None)]
+- OneRowRelation
at
org.apache.spark.sql.catalyst.analysis.package$AnalysisErrorAt.failAnalysis(package.scala:42)
```
This PR can fix this query, too.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
