[
https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16097166#comment-16097166
]
Sean Owen commented on SPARK-14643:
-----------------------------------
[~joshrosen] so we've always assumed that Scala versions aren't mutually
source-compatible. There are other versions of this ambiguity problem in the
Scala code, for Scala callers, that crop up in 2.12, but, I think we accept
those are a given for Scala version differences. 2.12 is already going to be
source incompatible, I think.
I see the point here is Java callers, who don't expect Scala version issues to
matter. But the Scala compiler changes have always potentially had implications
for Java callers, and here it's not just the ambiguity with lambdas, but the
fact the scalac will emit different interfaces in certain cases. The question
is here is whether source compatibility can be maintained, not binary, right?
Maintaining source compatibility for Java seems to mean even more API changes.
I'm just wondering out loud whether it's worth it? I mean, a Java-only app that
doesn't care should just keep using a Scala 2.11 build to insulate from all
Scala changes.
Mostly I'm trying to figure out whether it's realistic to introduce 2.12
support in 2.3.0, which would mean no API changes that would affect Scala 2.11,
and the remedy here would affect the 2.11 build I believe. (Unless we get into
different source code versions for different builds, which seems too much.)
> Remove overloaded methods which become ambiguous in Scala 2.12
> --------------------------------------------------------------
>
> Key: SPARK-14643
> URL: https://issues.apache.org/jira/browse/SPARK-14643
> Project: Spark
> Issue Type: Sub-task
> Components: Build, Project Infra
> Reporter: Josh Rosen
> Assignee: Josh Rosen
>
> Spark 1.x's Dataset API runs into subtle source incompatibility problems for
> Java 8 and Scala 2.12 users when Spark is built against Scala 2.12. In a
> nutshell, the current API has overloaded methods whose signatures are
> ambiguous when resolving calls that use the Java 8 lambda syntax (only if
> Spark is build against Scala 2.12).
> This issue is somewhat subtle, so there's a full writeup at
> https://docs.google.com/document/d/1P_wmH3U356f079AYgSsN53HKixuNdxSEvo8nw_tgLgM/edit?usp=sharing
> which describes the exact circumstances under which the current APIs are
> problematic. The writeup also proposes a solution which involves the removal
> of certain overloads only in Scala 2.12 builds of Spark and the introduction
> of implicit conversions for retaining source compatibility.
> We don't need to implement any of these changes until we add Scala 2.12
> support since the changes must only be applied when building against Scala
> 2.12 and will be done via traits + shims which are mixed in via
> per-Scala-version source directories (like how we handle the
> Scala-version-specific parts of the REPL). For now, this JIRA acts as a
> placeholder so that the parent JIRA reflects the complete set of tasks which
> need to be finished for 2.12 support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
