[ https://issues.apache.org/jira/browse/SPARK-4819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074001#comment-15074001 ]
Sean Owen commented on SPARK-4819: ---------------------------------- OK, I wrestled with this a couple hours and think this will fail: the problem is, adding the type constraint ultimately means most every signature in the Java APIs has to include the constraint. Then this starts to cause funny problems: you can't have the RDD.toJavaRDD method anymore since it can't guarantee the bound. Eventually the only unfixable problem I encountered had to do with the workaround for Java varags in the union() method. This hackery to work around scalac issues just won't fly; the type bound on the Scala code extending the Java class isn't allowed. Options seem to be: - Push further ahead and drop the varargs version of JavaSparkContext.union() as collateral damage and see what other surprises are in store - Copy and paste the Optional class from Guava and make it an official Spark API class - Require Java 8 and use its version - ... copy and paste Java 8's version > Remove Guava's "Optional" from public API > ----------------------------------------- > > Key: SPARK-4819 > URL: https://issues.apache.org/jira/browse/SPARK-4819 > Project: Spark > Issue Type: Sub-task > Components: Spark Core > Affects Versions: 1.2.0 > Reporter: Marcelo Vanzin > Assignee: Sean Owen > > Filing this mostly so this isn't forgotten. Spark currently exposes Guava > types in its public API (the {{Optional}} class is used in the Java > bindings). This makes it hard to properly hide Guava from user applications, > and makes mixing different Guava versions with Spark a little sketchy (even > if things should work, since those classes are pretty simple in general). > Since this changes the public API, it has to be done in a release that allows > such breakages. But it would be nice to at least have a transition plan for > deprecating the affected APIs. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org