[
https://issues.apache.org/jira/browse/SPARK-37560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-37560.
----------------------------------
Resolution: Duplicate
> equality comparison for map type field
> --------------------------------------
>
> Key: SPARK-37560
> URL: https://issues.apache.org/jira/browse/SPARK-37560
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.4.0
> Reporter: omar
> Priority: Major
>
> hi
> where I try to execute this kind of qury
>
> {code:java}
> SELECT * FROM myTable where myMapField is null {code}
> I get this king of error
>
> {code:java}
> org.apache.spark.sql.AnalysisException: Cannot have map type columns in
> DataFrame which calls set operations(intersect, except, etc.), but the type
> of column is map ... {code}
> According to this link
> [https://github.com/apache/spark/blob/branch-2.4/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L406]
>
>
> would it be possible to assign a default comparaison function for Map type ?
> i'm working on spark 2.4, and it seams that this issue happens with version
> 3.2 too (same comment in 3.2 branch)
>
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
