Re: mvn or sbt for studying and developing Spark?

2014-11-17 Thread Nicholas Chammas
The docs on using sbt are here: https://github.com/apache/spark/blob/master/docs/building-spark.md#building-with-sbt They'll be published with 1.2.0 presumably. On 2014년 11월 17일 (월) at 오후 2:49 Michael Armbrust wrote: > > > > * I moved from sbt to maven in June specifically due to Andrew Or's > >

Re: mvn or sbt for studying and developing Spark?

2014-11-17 Thread Michael Armbrust
> > * I moved from sbt to maven in June specifically due to Andrew Or's > describing mvn as the default build tool. Developers should keep in mind > that jenkins uses mvn so we need to run mvn before submitting PR's - even > if sbt were used for day to day dev work > To be clear, I think that the

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Mark Hamstra
on tool of > make-distribution.sh changed from sbt (in spark-0.9) to mvn(in spark-1.0). > > > > Cheers, > > Yiming > > > > 发件人: Dinesh J. Weerakkody [mailto:dineshjweerakk...@gmail.com] > 发送时间: 2014年11月16日 10:58 > 收件人: sdi...@gmail.com > 抄送: dev@spark.apache.o

re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Yiming (John) Zhang
sbt (in spark-0.9) to mvn(in spark-1.0). Cheers, Yiming 发件人: Dinesh J. Weerakkody [mailto:dineshjweerakk...@gmail.com] 发送时间: 2014年11月16日 10:58 收件人: sdi...@gmail.com 抄送: dev@spark.apache.org 主题: Re: mvn or sbt for studying and developing Spark? Hi Yiming, I believe that both SBT and MVN

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Mark Hamstra
Ok, strictly speaking, that's equivalent to your second class of examples, "development console", not the first "sbt console" On Sun, Nov 16, 2014 at 1:47 PM, Mark Hamstra wrote: > The console mode of sbt (just run >> sbt/sbt and then a long running console session is started that will >> accept

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Patrick Wendell
Neither is strictly optimal which is why we ended up supporting both. Our reference build for packaging is Maven so you are less likely to run into unexpected dependency issues, etc. Many developers use sbt as well. It's somewhat religion and the best thing might be to try both and see which you pr

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Mark Hamstra
> > The console mode of sbt (just run > sbt/sbt and then a long running console session is started that will accept > further commands) is great for building individual subprojects or running > single test suites. In addition to being faster since its a long running > JVM, its got a lot of nice fe

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Stephen Boesch
HI Michael, That insight is useful. Some thoughts: * I moved from sbt to maven in June specifically due to Andrew Or's describing mvn as the default build tool. Developers should keep in mind that jenkins uses mvn so we need to run mvn before submitting PR's - even if sbt were used for day to

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Sean Owen
Yeah, my comment was mostly reflecting the fact that mvn is what creates the releases and is the 'build of reference', from which the SBT build is generated. The docs were recently changed to suggest that Maven is the default build and SBT is for advanced users. I find Maven plays nicer with IDEs,

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Michael Armbrust
I'm going to have to disagree here. If you are building a release distribution or integrating with legacy systems then maven is probably the correct choice. However most of the core developers that I know use sbt, and I think its a better choice for exploration and development overall. That said,

Re: mvn or sbt for studying and developing Spark?

2014-11-16 Thread Dinesh J. Weerakkody
Hi Stephen and Sean, Thanks for correction. On Sun, Nov 16, 2014 at 12:28 PM, Sean Owen wrote: > No, the Maven build is the main one. I would use it unless you have a > need to use the SBT build in particular. > On Nov 16, 2014 2:58 AM, "Dinesh J. Weerakkody" < > dineshjweerakk...@gmail.com> w

Re: mvn or sbt for studying and developing Spark?

2014-11-15 Thread Sean Owen
No, the Maven build is the main one. I would use it unless you have a need to use the SBT build in particular. On Nov 16, 2014 2:58 AM, "Dinesh J. Weerakkody" wrote: > Hi Yiming, > > I believe that both SBT and MVN is supported in SPARK, but SBT is preferred > (I'm not 100% sure about this :) ).

Re: mvn or sbt for studying and developing Spark?

2014-11-15 Thread Dinesh J. Weerakkody
Hi Yiming, I believe that both SBT and MVN is supported in SPARK, but SBT is preferred (I'm not 100% sure about this :) ). When I'm using MVN I got some build failures. After that used SBT and works fine. You can go through these discussions regarding SBT vs MVN and learn pros and cons of both [1

mvn or sbt for studying and developing Spark?

2014-11-15 Thread Yiming (John) Zhang
Hi, I am new in developing Spark and my current focus is about co-scheduling of spark tasks. However, I am confused with the building tools: sometimes the documentation uses mvn but sometimes uses sbt. So, my question is that which one is the preferred tool of Spark community? And what's t