[ https://issues.apache.org/jira/browse/PIO-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15901189#comment-15901189 ]
ASF GitHub Bot commented on PIO-30: ----------------------------------- Github user shimamoto commented on the issue: https://github.com/apache/incubator-predictionio/pull/345 Hi @dszeto My suggestion is based on #352. This is what I see moving forward. - Define the versions of Scala to build against in the `crossScalaVersions` setting. - I’d be happy if it would be possible to deprecate Spark 1. We keep using the class (especially SQLContext) that keeping for backward compatibility for a while. - Or the code that depends on version of Spark makes it abstract, and separate build. The jar that depends on Spark 1 is in lib/spark and the other is in lib/extra. In order to use Spark 2, one would replaced like ES5. Note that this is just my imagination for now. Thus, I would like to try this out if this suggestion takes a turn for the better. For that, I think I need to some more time. Do you have any thoughts? > Cross build for different versions of scala and spark > ----------------------------------------------------- > > Key: PIO-30 > URL: https://issues.apache.org/jira/browse/PIO-30 > Project: PredictionIO > Issue Type: Improvement > Reporter: Marcin Ziemiński > Assignee: Chan > Fix For: 0.11.0 > > > The present version of Scala is 2.10 and Spark is 1.4, which is quite old. > With Spark 2.0.0 come many performance improvements and features, that people > will definitely like to add to their templates. I am also aware that past > cannot be ignored and simply dumping 1.x might not be an option for other > users. > I propose setting up a crossbuild in sbt to build with scala 2.10 and Spark > 1.6 and a separate one for Scala 2.11 and Spark 2.0. Most of the files will > be consistent between versions including API. The problematic ones will be > divided between additional source directories: src/main/scala-2.10/ and > src/main/scala-2.11/. The dockerized tests should also take the two versions > into consideration -- This message was sent by Atlassian JIRA (v6.3.15#6346)