[
https://issues.apache.org/jira/browse/MAHOUT-1653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380992#comment-14380992
]
Sean Owen commented on MAHOUT-1653:
-----------------------------------
I haven't been following this, but {{rddToSequenceFileRDDFunctions}} is not a
method you should be calling. The non-deprecated version is an implicit.
At a macro level the only 'official' way to invoke Spark stuff is via the
spark-submit script. That said there's nothing stopping you from making a
{{SparkContext}} and configuring it and going. What you miss out on, and may
have to duplicate, is a lot of the classpath stuff that the scripts take care
of.
1.4 has a real programmatic "Launcher" API.
I don't know of particular problems with Snappy except that there was some
problem that meant Spark had to use Snappy 1.1.x+. Otherwise should be about
the same to configure and use, which does mean setting a library path so it
can't find native libs.
Some of this may be caused by doing some things more directly into Spark and so
needing to track upstream changes to keep up. This isn't a stable API being
called. I don't know of a good reason you shouldn't be able to move to 1.3,
modulo the awful weirdness that is Guava shading and classpath... which I have
always dodged by just using Guava 14 too.
> Spark 1.3
> ---------
>
> Key: MAHOUT-1653
> URL: https://issues.apache.org/jira/browse/MAHOUT-1653
> Project: Mahout
> Issue Type: Dependency upgrade
> Affects Versions: 0.10.0
> Reporter: Andrew Musselman
> Assignee: Andrew Palumbo
> Priority: Critical
>
> Support Spark 1.3
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
