[ https://issues.apache.org/jira/browse/SPARK-32756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen updated SPARK-32756: --------------------------------- Priority: Minor (was: Major) > Fix CaseInsensitiveMap in Scala 2.13 > ------------------------------------ > > Key: SPARK-32756 > URL: https://issues.apache.org/jira/browse/SPARK-32756 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 3.1.0 > Reporter: Karol Chmist > Priority: Minor > > > "Spark SQL" module doesn't compile in Scala 2.13: > {code:java} > [info] Compiling 26 Scala sources to > /home/karol/workspace/open-source/spark/sql/core/target/scala-2.13/classes... > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala:121: > value += is not a member of > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] Expression does not convert to assignment because: > [error] type mismatch; > [error] found : scala.collection.immutable.Map[String,String] > [error] required: > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] expansion: this.extraOptions = > this.extraOptions.+(key.$minus$greater(value)) > [error] this.extraOptions += (key -> value) > [error] ^ > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala:132: > value += is not a member of > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] Expression does not convert to assignment because: > [error] type mismatch; > [error] found : scala.collection.immutable.Map[String,String] > [error] required: > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] expansion: this.extraOptions = > this.extraOptions.+(key.$minus$greater(value)) > [error] this.extraOptions += (key -> value) > [error] ^ > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala:294: > value += is not a member of > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] Expression does not convert to assignment because: > [error] type mismatch; > [error] found : scala.collection.immutable.Map[String,String] > [error] required: > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] expansion: this.extraOptions = > this.extraOptions.+("path".$minus$greater(path)) > [error] Error occurred in an application involving default arguments. > [error] this.extraOptions += ("path" -> path) > [error] ^ > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala:317: > type mismatch; > [error] found : Iterable[(String, String)] > [error] required: java.util.Map[String,String] > [error] Error occurred in an application involving default arguments. > [error] val dsOptions = new CaseInsensitiveStringMap(options.asJava) > [error] ^ > [info] Iterable[(String, String)] <: java.util.Map[String,String]? > [info] false > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala:412: > value += is not a member of > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] Expression does not convert to assignment because: > [error] type mismatch; > [error] found : scala.collection.immutable.Map[String,String] > [error] required: > org.apache.spark.sql.catalyst.util.CaseInsensitiveMap[String] > [error] expansion: DataFrameWriter.this.extraOptions = > DataFrameWriter.this.extraOptions.+(DataSourceUtils.PARTITIONING_COLUMNS_KEY.$minus$greater(DataSourceUtils.encodePartitioningColumns(columns))) > [error] extraOptions += (DataSourceUtils.PARTITIONING_COLUMNS_KEY -> > [error] ^ > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFiltersBase.scala:85: > type mismatch; > [error] found : > scala.collection.MapView[String,OrcFiltersBase.this.OrcPrimitiveField] > [error] required: Map[String,OrcFiltersBase.this.OrcPrimitiveField] > [error] CaseInsensitiveMap(dedupPrimitiveFields) > [error] ^ > [info] scala.collection.MapView[String,OrcFiltersBase.this.OrcPrimitiveField] > <: Map[String,OrcFiltersBase.this.OrcPrimitiveField]? > [info] false > [error] > /home/karol/workspace/open-source/spark/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/FileDataSourceV2.scala:64: > type mismatch; > [error] found : Iterable[(String, String)] > [error] required: java.util.Map[String,String] > [error] new CaseInsensitiveStringMap(withoutPath.asJava) > [error] ^ > [info] Iterable[(String, String)] <: java.util.Map[String,String]? > [error] 7 errors found{code} > > The + function in CaseInsensitiveStringMap missing, resulting in {{+}} from > standard Map being used, which returns Map instead of > CaseInsensitiveStringMap. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org