[
https://issues.apache.org/jira/browse/SPARK-14643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16097169#comment-16097169
]
Josh Rosen commented on SPARK-14643:
------------------------------------
[~srowen], I just posted a comment about this over at
https://github.com/apache/spark/pull/18645#discussion_r128892346. Quoting here:
{quote}
I think this is primarily going to be an issue for end users who want to use an
existing source tree to cross-compile for Scala 2.10, 2.11, and 2.12. Thus the
pain of the source incompatibility would mostly be felt by library/package
maintainers but it can be worked around as long as there's at least some common
subset which is source compatible across all of those versions.
A similar ambiguity issue exists for people writing Spark code in Java 8 and
who use lambda syntax. The number of such users must also upgrade to Scala 2.12
is probably fairly small and the number of those users who have a requirement
to cross-build their Java 8 lambda syntax Spark code against 2.11 and 2.12 is
probably even smaller, so maybe this isn't a huge deal to accept the ambiguity
and require those users to do a bit of work when upgrading.
{quote}
> Remove overloaded methods which become ambiguous in Scala 2.12
> --------------------------------------------------------------
>
> Key: SPARK-14643
> URL: https://issues.apache.org/jira/browse/SPARK-14643
> Project: Spark
> Issue Type: Sub-task
> Components: Build, Project Infra
> Reporter: Josh Rosen
> Assignee: Josh Rosen
>
> Spark 1.x's Dataset API runs into subtle source incompatibility problems for
> Java 8 and Scala 2.12 users when Spark is built against Scala 2.12. In a
> nutshell, the current API has overloaded methods whose signatures are
> ambiguous when resolving calls that use the Java 8 lambda syntax (only if
> Spark is build against Scala 2.12).
> This issue is somewhat subtle, so there's a full writeup at
> https://docs.google.com/document/d/1P_wmH3U356f079AYgSsN53HKixuNdxSEvo8nw_tgLgM/edit?usp=sharing
> which describes the exact circumstances under which the current APIs are
> problematic. The writeup also proposes a solution which involves the removal
> of certain overloads only in Scala 2.12 builds of Spark and the introduction
> of implicit conversions for retaining source compatibility.
> We don't need to implement any of these changes until we add Scala 2.12
> support since the changes must only be applied when building against Scala
> 2.12 and will be done via traits + shims which are mixed in via
> per-Scala-version source directories (like how we handle the
> Scala-version-specific parts of the REPL). For now, this JIRA acts as a
> placeholder so that the parent JIRA reflects the complete set of tasks which
> need to be finished for 2.12 support.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
