Thanks guys; patch is up here: https://issues.apache.org/jira/browse/CRUNCH-384
Micah, do you want to take a pass through this one? J On Mon, Apr 28, 2014 at 10:30 PM, Som Satpathy <[email protected]>wrote: > +1 for option 2. > > Thanks, > Som > > > On Fri, Apr 25, 2014 at 1:45 PM, Josh Wills <[email protected]> wrote: > >> Hey all, >> >> I've been working on doing a new release candidate, and I think that >> we're going to need to upgrade to Spark 0.9.1 in order for Crunch-on-Spark >> to work in Spark standalone/Spark-on-YARN on top of Hadoop, as opposed to >> the local mode that works now. There are a couple of implications to this, >> but the big one is that Spark 0.9.1 is only developed against Scala 2.10, >> which would mean that we would need to switch over to 2.10 for at least >> crunch-spark, and it would seem much easier to me for us to upgrade Scrunch >> to 2.10 as well. >> >> Couple of options here off the top of my head: >> 1) Do the upgrade to Scala 2.10/Spark 0.9.1 in 0.10.0, but not 0.8.3. >> 2) Do the upgrade in both 0.8.3 and 0.10.0. >> 3) Only upgrade Spark to 2.10 (in either 0.8.3 or 0.10.0), but still have >> Scrunch build against 2.9.3. >> >> I'm in favor of #2 myself, but am open to suggestion here, and would >> especially like to know if anyone feels strongly about staying on 2.9.3 for >> any reason. >> >> Thanks! >> Josh >> >> Director of Data Science >> Cloudera <http://www.cloudera.com> >> Twitter: @josh_wills <http://twitter.com/josh_wills> >> > > -- Director of Data Science Cloudera <http://www.cloudera.com> Twitter: @josh_wills <http://twitter.com/josh_wills>
