[
https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074001#comment-15074001
]
Sean Owen commented on SPARK-4819:
----------------------------------
OK, I wrestled with this a couple hours and think this will fail: the problem
is, adding the type constraint ultimately means most every signature in the
Java APIs has to include the constraint. Then this starts to cause funny
problems: you can't have the RDD.toJavaRDD method anymore since it can't
guarantee the bound. Eventually the only unfixable problem I encountered had to
do with the workaround for Java varags in the union() method. This hackery to
work around scalac issues just won't fly; the type bound on the Scala code
extending the Java class isn't allowed.
Options seem to be:
- Push further ahead and drop the varargs version of JavaSparkContext.union()
as collateral damage and see what other surprises are in store
- Copy and paste the Optional class from Guava and make it an official Spark
API class
- Require Java 8 and use its version
- ... copy and paste Java 8's version
> Remove Guava's "Optional" from public API
> -----------------------------------------
>
> Key: SPARK-4819
> URL: https://issues.apache.org/jira/browse/SPARK-4819
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 1.2.0
> Reporter: Marcelo Vanzin
> Assignee: Sean Owen
>
> Filing this mostly so this isn't forgotten. Spark currently exposes Guava
> types in its public API (the {{Optional}} class is used in the Java
> bindings). This makes it hard to properly hide Guava from user applications,
> and makes mixing different Guava versions with Spark a little sketchy (even
> if things should work, since those classes are pretty simple in general).
> Since this changes the public API, it has to be done in a release that allows
> such breakages. But it would be nice to at least have a transition plan for
> deprecating the affected APIs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
