[
https://issues.apache.org/jira/browse/SPARK-29292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17096893#comment-17096893
]
Guillaume Martres commented on SPARK-29292:
-------------------------------------------
If it's immutable it's fine yeah, but it seems that spark internally uses a
bunch of ArrayBuffer which do need to be copied to be made into a scala.Seq
now. On top of that, users of Spark might also have to add a bunch of
potentially-copying .toSeq to call Spark methods. For example I had some code
that did `sparkContext.parallelize(rdd.take(1000))` which still compiles but
with a deprecation warning because take returns an Array:
> warning: method copyArrayToImmutableIndexedSeq in class LowPriorityImplicits2
> is deprecated (since 2.13.0): Implicit conversions from Array to
> immutable.IndexedSeq are implemented by copying; Use the more efficient
> non-copying ArraySeq.unsafeWrapArray or an explicit toIndexedSeq call
So depending on how common this sort of things is, it might make sense to
change SparkContext#parallelize and others to take a scala.collection.Seq
instead of a Seq.
> Fix internal usages of mutable collection for Seq in 2.13
> ---------------------------------------------------------
>
> Key: SPARK-29292
> URL: https://issues.apache.org/jira/browse/SPARK-29292
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.0.0
> Reporter: Sean R. Owen
> Assignee: Sean R. Owen
> Priority: Minor
>
> Kind of related to https://issues.apache.org/jira/browse/SPARK-27681, but a
> simpler subset.
> In 2.13, a mutable collection can't be returned as a
> {{scala.collection.Seq}}. It's easy enough to call .toSeq on these as that
> still works on 2.12.
> {code}
> [ERROR] [Error]
> /Users/seanowen/Documents/spark_2.13/core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala:467:
> type mismatch;
> found : Seq[String] (in scala.collection)
> required: Seq[String] (in scala.collection.immutable)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
