[
https://issues.apache.org/jira/browse/SPARK-27681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16838744#comment-16838744
]
Sean Owen commented on SPARK-27681:
-----------------------------------
OK, take it down a notch; we are largely in agreement, but there's a little
more to it.
Right, so if all Spark's {{scala.Seq}} in APIs switch their implementation,
then so do callers' {{scala.Seq}}, so who cares? The problem is that Spark, and
user code, _do_ sometimes use Seqs that aren't known to be immutable. Your
point is specifically that {{def foo(): scala.Seq}} and {{val a: scala.Seq =
foo()}} work fine in 2.12 and 2.13, but changing to {{def foo():
scala.collection.Seq}} works in 2.12 but not 2.13, unless you use the type more
explicitly. Yeah, let's avoid that.
There's a mirror-image issue. A Spark method that returns a non-immutable Seq
as {{scala.Seq}} works now but won't compile in 2.13. OK, but that's Spark's
problem; we either change the Spark API or change its implementation, though I
doubt it'll be possible in all cases.
But, user code that passes a non-immutable Seq to a Spark API that accepts
{{scala.Seq}} now will no longer compile in 2.13. That is if {{def foo(a:
scala.Seq)}}, it works with any Seq in 2.12 but fails to compile for
{{scala.collection.Seq}} arguments in 2.13. Whereas {{def foo(a:
scala.collection.Seq)}} continues to work in both.
Fixing the type to {{scala.collection.Seq}} isn't so much about making sure
it's the same type in 2.12 and 2.13, but fixing this at a type that can accept
any Seq as it does already, in 2.13. The cases that may have to change are the
covariant or contravariant cases, I forget which is which. Definitely not all
of them.
Which ones? I'd punt that to later, but I think it'll be method arguments that
are Seqs and Spark methods whose return type don't compile otherwise in 2.13.
Oh, and the description definitely has to be updated, I'll do that.
> Use scala.collection.Seq explicitly instead of scala.Seq alias
> --------------------------------------------------------------
>
> Key: SPARK-27681
> URL: https://issues.apache.org/jira/browse/SPARK-27681
> Project: Spark
> Issue Type: Sub-task
> Components: ML, MLlib, Spark Core, SQL, Structured Streaming
> Affects Versions: 3.0.0
> Reporter: Sean Owen
> Assignee: Sean Owen
> Priority: Major
>
> {{scala.Seq}} is widely used in the code, and is an alias for
> {{scala.collection.Seq}} in Scala 2.12. It will become an alias for
> {{scala.collection.immutable.Seq}} in Scala 2.13. To avoid API changes, we
> should simply explicit import and use {{scala.collection.Seq}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
