[
https://issues.apache.org/jira/browse/MAHOUT-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849507#comment-15849507
]
ASF GitHub Bot commented on MAHOUT-1894:
----------------------------------------
GitHub user rawkintrevo opened a pull request:
https://github.com/apache/mahout/pull/271
[MAHOUT-1894] Add Support for Spark 2.x
As long as we're sticking to Scala 2.10, running mahout on spark 2.x is
simply a matter of
`mvn clean package -Dspark.version=2.0.2`
or
`mvn clean package -Dspark.version=2.1.0`
The trouble comes with the shell...
I checked Apache Zeppelin to see how they handle multiple spark/scala
versions...
[a brief preview of the descent into hell that is having a shell that
handles multiple spark/scala
versions](https://github.com/apache/zeppelin/blob/master/spark/src/main/java/org/apache/zeppelin/spark/SparkInterpreter.java)
So I took an alternate root. I dropped the Mahout shell all together,
changed the mahout bin file to load the spark shell directly, and pass a scala
script that takes care of our imports.
When building there is a single deprecation warning regarding the
sqlContext and how it is created in the spark-bindings.
I think we should add binaries for Spark 2.0 and Spark 2.1 as a matter of
convenience and the Zeppelin integration.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rawkintrevo/mahout mahout-1894
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/mahout/pull/271.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #271
----
commit 867cdd0c04d629eaf44a0e2031f447d03bf67bcc
Author: rawkintrevo <[email protected]>
Date: 2017-02-02T06:18:21Z
MAHOUT-1894 Add support for spark 2.x
MAHOUT-1894 Add support for spark 2.x
----
> Add support for Spark 2x backend
> --------------------------------
>
> Key: MAHOUT-1894
> URL: https://issues.apache.org/jira/browse/MAHOUT-1894
> Project: Mahout
> Issue Type: Task
> Components: spark
> Affects Versions: 0.13.0
> Reporter: Suneel Marthi
> Priority: Critical
> Fix For: 1.0.0, 0.13.0, 0.14.0
>
>
> add support for Spark 2.x as backend execution engine.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
