There is also an alternative called 'sbt-onejar" we can look at.
On Thu, Jul 18, 2013 at 10:48 AM, Matei Zaharia <[email protected]>wrote: > Thanks for the feedback. It looks like there are more advantages to Maven > than I was originally thinking of -- specifically, the better dependency > resolution and assembly construction. (SBT assembly just takes all the JARs > in lib_managed and packages them together unfortunately, which means you > sometimes get multiple versions of the same artifact if you aren't very > careful with exclusion rules). I think what we'll do is to wait until we > see whether we can have a single Spark artifact that works with any Hadoop > version, and go back to the build system issue then. > > Matei > > On Jul 16, 2013, at 4:37 PM, Ryan LeCompte <[email protected]> wrote: > > +1 for SBT. It's pretty much the de facto standard for mainstream Scala > projects. > > > On Tue, Jul 16, 2013 at 4:19 PM, Evan Chan <[email protected]> wrote: > >> If the APIs for those libraries such as Akka stay the same, you don't >> need different branches. In SBT you can easily support two different sets >> of deps depending on which Scala version you are building.... not sure if >> you could do that with Maven. >> >> Here are the main differences that would require a different branch: >> - if you are taking advantage of Akka Cluster, or features exclusive to >> newer releases >> - The Akka futures API changed packages between Scala 2.9.1/2 and Scala >> 2.9.3/2.10. However, since the project upgraded to Scala 2.9.3, it should >> migrate all use of futures to the scala.concurrent.* namespace to avoid >> more code changes down the line. >> >> As far as SBT vs Maven vs Gradle etc..... >> - I personally think SBT's "console" REPL with all dependencies in the >> classpath is super valuable for development, ad hoc testing, trying new >> ideas out. Not sure if Maven has that. >> - SBT also has triggered recompilation/testing, which is again very >> valuable for some of us >> - Gradle has a easier to understand DSL, but it seems once you build a >> complex enough system, it is no easier to maintain than SBT >> - Maven is much better integrated into most IDEs, but SBT is far superior >> for dev workflow if you don't use one of the major IDEs or are a command >> line person. >> >> -Evan >> >> >> On Tuesday, July 16, 2013 1:37:41 PM UTC-7, Matei Zaharia wrote: >> >>> Unfortunately, we'll probably have to have different branches of Spark >>> for different Scala versions, because there are also other libraries we >>> depend on (e.g. Akka) that have separate versions for Scala 2.10. You can >>> actually find a Scala 2.10 port of Spark in the scala-2.10 branch on >>> GitHub. >>> >>> Matei >>> >>> >>> On Jul 16, 2013, at 1:33 PM, Cody Koeninger <[email protected]> >>> wrote: >>> >>> > If you're looking at consolidating build systems, I'd ask to consider >>> ease of cross-publishing for different Scala versions. My instinct is that >>> sbt will be less troublesome in that regard (although as I understand it, >>> the changes to the repl may present a problem). >>> > >>> > We're needing to use 2.10 for a project, so I'd be happy to put in >>> some work on the issue. >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Spark Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > > -- > You received this message because you are subscribed to the Google Groups > "Spark Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "Spark Developers" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/spark-developers/OxL268v0-Qs/unsubscribe > . > To unsubscribe from this group and all its topics, send an email to > [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- -- Evan Chan Staff Engineer [email protected] | <http://www.ooyala.com/> <http://www.facebook.com/ooyala><http://www.linkedin.com/company/ooyala><http://www.twitter.com/ooyala>
