Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi Matei, Thanks for your response. I am using 0.8.1, and yes, It was using protobuf-2.5. Sorry I made a mistake before this email. I used -Phadoop2-yarn, so I don't find it using pb-2.5, actually, I should use -Pnew-yarn. Thank you Matei. On Mon, Dec 16, 2013 at 4:22 PM, Matei Zaharia wrote: > Are you using 0.8.1? It will build with protobuf 2.5 instead of 2.4 as > long as you make it depend on Hadoop 2.2. But make sure you build it with > SPARK_HADOOP_VERSION=2.2.0 or whatever. > > Spark 0.8.0 doesn’t support Hadoop 2.2 due to this issue. > > Matei > > On Dec 15, 2013, at 10:25 PM, Azuryy Yu wrote: > > > Maybe I am not give a clear description. I am runing Spark on yarn. > instead > > of Mesos. I just want build Spark with protobuf2.5. I am not care about > > Mesos. > > > > I've changed Spark pom.xml to probobuf2.5 manually. > > > > > > > > > > On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia >wrote: > > > >> Mesos will almost certainly compile fine with protobuf 2.5. The protobuf > >> compiler and binary format is forward-compatible across releases, it’s > just > >> the Java artifacts that aren’t. You’ll need to ask Mesos to provide a > >> version with protobuf 2.5, and use that with these versions of Hadoop. > >> > >> Matei > >> > >> On Dec 15, 2013, at 7:00 PM, Liu, Raymond > wrote: > >> > >>> That issue is for 0.9's solution. > >>> > >>> And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, > >> protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine > >> with hadoop 2.2 > >>> And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run > upon > >> mesos... strange combination, I am not sure, might have problem. If have > >> problem, you might need to build mesos against 2.5.0, I don't test > that, if > >> you got time, mind take a test? > >>> > >>> Best Regards, > >>> Raymond Liu > >>> > >>> > >>> -Original Message- > >>> From: Liu, Raymond [mailto:raymond@intel.com] > >>> Sent: Monday, December 16, 2013 10:48 AM > >>> To: dev@spark.incubator.apache.org > >>> Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > >>> > >>> Hi Azuryy > >>> > >>> Please Check https://spark-project.atlassian.net/browse/SPARK-995 for > >> this protobuf version issue > >>> > >>> Best Regards, > >>> Raymond Liu > >>> > >>> -Original Message- > >>> From: Azuryy Yu [mailto:azury...@gmail.com] > >>> Sent: Monday, December 16, 2013 10:30 AM > >>> To: dev@spark.incubator.apache.org > >>> Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > >>> > >>> Hi here, > >>> Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some > >> uncompatable API between these two versions. > >>> Hadoop-2.x using protobuf-2.5.0 > >>> > >>> > >>> but if some guys want to run Spark on mesos, then mesos using > >>> protobuf-2.4.1 currently. so we may discuss here for a better solution. > >>> > >>> > >>> > >>> On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: > >>> > >>>> Thanks Patrick. > >>>> On 16 Dec 2013 02:43, "Patrick Wendell" wrote: > >>>> > >>>>> You can checkout the docs mentioned in the vote thread. There is also > >>>>> a pre-build binary for hadoop2 that is compiled for YARN 2.2 > >>>>> > >>>>> - Patrick > >>>>> > >>>>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu > wrote: > >>>>>> yarn 2.2, not yarn 0.22, I am so sorry. > >>>>>> > >>>>>> > >>>>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu > >> wrote: > >>>>>> > >>>>>>> Hi, > >>>>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release > note? > >>>>>>> Thanks. > >>>>>>> > >>>>>>> > >>>>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < > >>>>> henry.sapu...@gmail.com>wrote: > >>>>>>> > >>>>>>>> Yeah seems like it. He was ok with our prev
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Are you using 0.8.1? It will build with protobuf 2.5 instead of 2.4 as long as you make it depend on Hadoop 2.2. But make sure you build it with SPARK_HADOOP_VERSION=2.2.0 or whatever. Spark 0.8.0 doesn’t support Hadoop 2.2 due to this issue. Matei On Dec 15, 2013, at 10:25 PM, Azuryy Yu wrote: > Maybe I am not give a clear description. I am runing Spark on yarn. instead > of Mesos. I just want build Spark with protobuf2.5. I am not care about > Mesos. > > I've changed Spark pom.xml to probobuf2.5 manually. > > > > > On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia wrote: > >> Mesos will almost certainly compile fine with protobuf 2.5. The protobuf >> compiler and binary format is forward-compatible across releases, it’s just >> the Java artifacts that aren’t. You’ll need to ask Mesos to provide a >> version with protobuf 2.5, and use that with these versions of Hadoop. >> >> Matei >> >> On Dec 15, 2013, at 7:00 PM, Liu, Raymond wrote: >> >>> That issue is for 0.9's solution. >>> >>> And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, >> protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine >> with hadoop 2.2 >>> And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon >> mesos... strange combination, I am not sure, might have problem. If have >> problem, you might need to build mesos against 2.5.0, I don't test that, if >> you got time, mind take a test? >>> >>> Best Regards, >>> Raymond Liu >>> >>> >>> -Original Message- >>> From: Liu, Raymond [mailto:raymond@intel.com] >>> Sent: Monday, December 16, 2013 10:48 AM >>> To: dev@spark.incubator.apache.org >>> Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) >>> >>> Hi Azuryy >>> >>> Please Check https://spark-project.atlassian.net/browse/SPARK-995 for >> this protobuf version issue >>> >>> Best Regards, >>> Raymond Liu >>> >>> -Original Message- >>> From: Azuryy Yu [mailto:azury...@gmail.com] >>> Sent: Monday, December 16, 2013 10:30 AM >>> To: dev@spark.incubator.apache.org >>> Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) >>> >>> Hi here, >>> Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some >> uncompatable API between these two versions. >>> Hadoop-2.x using protobuf-2.5.0 >>> >>> >>> but if some guys want to run Spark on mesos, then mesos using >>> protobuf-2.4.1 currently. so we may discuss here for a better solution. >>> >>> >>> >>> On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: >>> >>>> Thanks Patrick. >>>> On 16 Dec 2013 02:43, "Patrick Wendell" wrote: >>>> >>>>> You can checkout the docs mentioned in the vote thread. There is also >>>>> a pre-build binary for hadoop2 that is compiled for YARN 2.2 >>>>> >>>>> - Patrick >>>>> >>>>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: >>>>>> yarn 2.2, not yarn 0.22, I am so sorry. >>>>>> >>>>>> >>>>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu >> wrote: >>>>>> >>>>>>> Hi, >>>>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? >>>>>>> Thanks. >>>>>>> >>>>>>> >>>>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < >>>>> henry.sapu...@gmail.com>wrote: >>>>>>> >>>>>>>> Yeah seems like it. He was ok with our prev release. >>>>>>>> Let's wait for his reply >>>>>>>> >>>>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote: >>>>>>>> >>>>>>>>> Henry - from that thread it looks like sebb's concern was >>>>>>>>> something different than this. >>>>>>>>> >>>>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >>>>>>>> henry.sapu...@gmail.com> >>>>>>>>> wrote: >>>>>>>>>> Hi Patrick, >>>>>>>>>> >>>>>>>>>> Yeap I agree, but technically ASF VOTE r
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Maybe I am not give a clear description. I am runing Spark on yarn. instead of Mesos. I just want build Spark with protobuf2.5. I am not care about Mesos. I've changed Spark pom.xml to probobuf2.5 manually. On Mon, Dec 16, 2013 at 2:02 PM, Matei Zaharia wrote: > Mesos will almost certainly compile fine with protobuf 2.5. The protobuf > compiler and binary format is forward-compatible across releases, it’s just > the Java artifacts that aren’t. You’ll need to ask Mesos to provide a > version with protobuf 2.5, and use that with these versions of Hadoop. > > Matei > > On Dec 15, 2013, at 7:00 PM, Liu, Raymond wrote: > > > That issue is for 0.9's solution. > > > > And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, > protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine > with hadoop 2.2 > > And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon > mesos... strange combination, I am not sure, might have problem. If have > problem, you might need to build mesos against 2.5.0, I don't test that, if > you got time, mind take a test? > > > > Best Regards, > > Raymond Liu > > > > > > -Original Message- > > From: Liu, Raymond [mailto:raymond....@intel.com] > > Sent: Monday, December 16, 2013 10:48 AM > > To: dev@spark.incubator.apache.org > > Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > > > > Hi Azuryy > > > > Please Check https://spark-project.atlassian.net/browse/SPARK-995 for > this protobuf version issue > > > > Best Regards, > > Raymond Liu > > > > -Original Message----- > > From: Azuryy Yu [mailto:azury...@gmail.com] > > Sent: Monday, December 16, 2013 10:30 AM > > To: dev@spark.incubator.apache.org > > Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > > > > Hi here, > > Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some > uncompatable API between these two versions. > > Hadoop-2.x using protobuf-2.5.0 > > > > > > but if some guys want to run Spark on mesos, then mesos using > > protobuf-2.4.1 currently. so we may discuss here for a better solution. > > > > > > > > On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: > > > >> Thanks Patrick. > >> On 16 Dec 2013 02:43, "Patrick Wendell" wrote: > >> > >>> You can checkout the docs mentioned in the vote thread. There is also > >>> a pre-build binary for hadoop2 that is compiled for YARN 2.2 > >>> > >>> - Patrick > >>> > >>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: > >>>> yarn 2.2, not yarn 0.22, I am so sorry. > >>>> > >>>> > >>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu > wrote: > >>>> > >>>>> Hi, > >>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? > >>>>> Thanks. > >>>>> > >>>>> > >>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < > >>> henry.sapu...@gmail.com>wrote: > >>>>> > >>>>>> Yeah seems like it. He was ok with our prev release. > >>>>>> Let's wait for his reply > >>>>>> > >>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote: > >>>>>> > >>>>>>> Henry - from that thread it looks like sebb's concern was > >>>>>>> something different than this. > >>>>>>> > >>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < > >>>>>> henry.sapu...@gmail.com> > >>>>>>> wrote: > >>>>>>>> Hi Patrick, > >>>>>>>> > >>>>>>>> Yeap I agree, but technically ASF VOTE release on source > >>>>>>>> only, > >>> there > >>>>>>>> even debate about it =), so putting it in the vote staging > >>> artifact > >>>>>>>> could confuse people because in our case we do package 3rd > >>>>>>>> party libraries in the binary jars. > >>>>>>>> > >>>>>>>> I have sent email to sebb asking clarification about his > >>>>>>>> concern > >>> in > >>>>>>>> general@ list. > >>>>>>>> > >>>>&
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Mesos will almost certainly compile fine with protobuf 2.5. The protobuf compiler and binary format is forward-compatible across releases, it’s just the Java artifacts that aren’t. You’ll need to ask Mesos to provide a version with protobuf 2.5, and use that with these versions of Hadoop. Matei On Dec 15, 2013, at 7:00 PM, Liu, Raymond wrote: > That issue is for 0.9's solution. > > And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, protobuf > is already using 2.5.0 instead of 2.4.1. so it will works fine with hadoop 2.2 > And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon > mesos... strange combination, I am not sure, might have problem. If have > problem, you might need to build mesos against 2.5.0, I don't test that, if > you got time, mind take a test? > > Best Regards, > Raymond Liu > > > -Original Message- > From: Liu, Raymond [mailto:raymond@intel.com] > Sent: Monday, December 16, 2013 10:48 AM > To: dev@spark.incubator.apache.org > Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > > Hi Azuryy > > Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this > protobuf version issue > > Best Regards, > Raymond Liu > > -Original Message- > From: Azuryy Yu [mailto:azury...@gmail.com] > Sent: Monday, December 16, 2013 10:30 AM > To: dev@spark.incubator.apache.org > Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) > > Hi here, > Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some > uncompatable API between these two versions. > Hadoop-2.x using protobuf-2.5.0 > > > but if some guys want to run Spark on mesos, then mesos using > protobuf-2.4.1 currently. so we may discuss here for a better solution. > > > > On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: > >> Thanks Patrick. >> On 16 Dec 2013 02:43, "Patrick Wendell" wrote: >> >>> You can checkout the docs mentioned in the vote thread. There is also >>> a pre-build binary for hadoop2 that is compiled for YARN 2.2 >>> >>> - Patrick >>> >>> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: >>>> yarn 2.2, not yarn 0.22, I am so sorry. >>>> >>>> >>>> On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: >>>> >>>>> Hi, >>>>> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? >>>>> Thanks. >>>>> >>>>> >>>>> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < >>> henry.sapu...@gmail.com>wrote: >>>>> >>>>>> Yeah seems like it. He was ok with our prev release. >>>>>> Let's wait for his reply >>>>>> >>>>>> On Saturday, December 14, 2013, Patrick Wendell wrote: >>>>>> >>>>>>> Henry - from that thread it looks like sebb's concern was >>>>>>> something different than this. >>>>>>> >>>>>>> On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >>>>>> henry.sapu...@gmail.com> >>>>>>> wrote: >>>>>>>> Hi Patrick, >>>>>>>> >>>>>>>> Yeap I agree, but technically ASF VOTE release on source >>>>>>>> only, >>> there >>>>>>>> even debate about it =), so putting it in the vote staging >>> artifact >>>>>>>> could confuse people because in our case we do package 3rd >>>>>>>> party libraries in the binary jars. >>>>>>>> >>>>>>>> I have sent email to sebb asking clarification about his >>>>>>>> concern >>> in >>>>>>>> general@ list. >>>>>>>> >>>>>>>> - Henry >>>>>>>> >>>>>>>> On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell < >>> pwend...@gmail.com >>>>>>> >>>>>>> wrote: >>>>>>>>> Hey Henry, >>>>>>>>> >>>>>>>>> One thing a lot of people do during the vote is test the >>> binaries and >>>>>>>>> make sure they work. This is really valuable. If you'd like >>>>>>>>> I >>> could >>>>>>>>> add a caveat to the vote thread explaining that we are only >
RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
That issue is for 0.9's solution. And if you mean for 0.8.1, when you build against hadoop 2.2 Yarn, protobuf is already using 2.5.0 instead of 2.4.1. so it will works fine with hadoop 2.2 And regarding on 0.8.1 you build against hadoop 2.2 Yarn, while run upon mesos... strange combination, I am not sure, might have problem. If have problem, you might need to build mesos against 2.5.0, I don't test that, if you got time, mind take a test? Best Regards, Raymond Liu -Original Message- From: Liu, Raymond [mailto:raymond@intel.com] Sent: Monday, December 16, 2013 10:48 AM To: dev@spark.incubator.apache.org Subject: RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi Azuryy Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version issue Best Regards, Raymond Liu -Original Message- From: Azuryy Yu [mailto:azury...@gmail.com] Sent: Monday, December 16, 2013 10:30 AM To: dev@spark.incubator.apache.org Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi here, Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions. Hadoop-2.x using protobuf-2.5.0 but if some guys want to run Spark on mesos, then mesos using protobuf-2.4.1 currently. so we may discuss here for a better solution. On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: > Thanks Patrick. > On 16 Dec 2013 02:43, "Patrick Wendell" wrote: > >> You can checkout the docs mentioned in the vote thread. There is also >> a pre-build binary for hadoop2 that is compiled for YARN 2.2 >> >> - Patrick >> >> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: >> > yarn 2.2, not yarn 0.22, I am so sorry. >> > >> > >> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: >> > >> >> Hi, >> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? >> >> Thanks. >> >> >> >> >> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < >> henry.sapu...@gmail.com>wrote: >> >> >> >>> Yeah seems like it. He was ok with our prev release. >> >>> Let's wait for his reply >> >>> >> >>> On Saturday, December 14, 2013, Patrick Wendell wrote: >> >>> >> >>> > Henry - from that thread it looks like sebb's concern was >> >>> > something different than this. >> >>> > >> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >> >>> henry.sapu...@gmail.com> >> >>> > wrote: >> >>> > > Hi Patrick, >> >>> > > >> >>> > > Yeap I agree, but technically ASF VOTE release on source >> >>> > > only, >> there >> >>> > > even debate about it =), so putting it in the vote staging >> artifact >> >>> > > could confuse people because in our case we do package 3rd >> >>> > > party libraries in the binary jars. >> >>> > > >> >>> > > I have sent email to sebb asking clarification about his >> >>> > > concern >> in >> >>> > > general@ list. >> >>> > > >> >>> > > - Henry >> >>> > > >> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell < >> pwend...@gmail.com >> >>> > >> >>> > wrote: >> >>> > >> Hey Henry, >> >>> > >> >> >>> > >> One thing a lot of people do during the vote is test the >> binaries and >> >>> > >> make sure they work. This is really valuable. If you'd like >> >>> > >> I >> could >> >>> > >> add a caveat to the vote thread explaining that we are only >> voting on >> >>> > >> the source. >> >>> > >> >> >>> > >> - Patrick >> >>> > >> >> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < >> >>> > henry.sapu...@gmail.com> wrote: >> >>> > >>> Actually we should be fine putting the binaries there as >> >>> > >>> long >> as the >> >>> > >>> VOTE is for the source. >> >>> > >>> >> >>> > >>> Let's verify with sebb in the general@ list about his concern. >> >>> > >>> >> >>>
RE: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi Azuryy Please Check https://spark-project.atlassian.net/browse/SPARK-995 for this protobuf version issue Best Regards, Raymond Liu -Original Message- From: Azuryy Yu [mailto:azury...@gmail.com] Sent: Monday, December 16, 2013 10:30 AM To: dev@spark.incubator.apache.org Subject: Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4) Hi here, Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions. Hadoop-2.x using protobuf-2.5.0 but if some guys want to run Spark on mesos, then mesos using protobuf-2.4.1 currently. so we may discuss here for a better solution. On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: > Thanks Patrick. > On 16 Dec 2013 02:43, "Patrick Wendell" wrote: > >> You can checkout the docs mentioned in the vote thread. There is also >> a pre-build binary for hadoop2 that is compiled for YARN 2.2 >> >> - Patrick >> >> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: >> > yarn 2.2, not yarn 0.22, I am so sorry. >> > >> > >> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: >> > >> >> Hi, >> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? >> >> Thanks. >> >> >> >> >> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < >> henry.sapu...@gmail.com>wrote: >> >> >> >>> Yeah seems like it. He was ok with our prev release. >> >>> Let's wait for his reply >> >>> >> >>> On Saturday, December 14, 2013, Patrick Wendell wrote: >> >>> >> >>> > Henry - from that thread it looks like sebb's concern was >> >>> > something different than this. >> >>> > >> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >> >>> henry.sapu...@gmail.com> >> >>> > wrote: >> >>> > > Hi Patrick, >> >>> > > >> >>> > > Yeap I agree, but technically ASF VOTE release on source >> >>> > > only, >> there >> >>> > > even debate about it =), so putting it in the vote staging >> artifact >> >>> > > could confuse people because in our case we do package 3rd >> >>> > > party libraries in the binary jars. >> >>> > > >> >>> > > I have sent email to sebb asking clarification about his >> >>> > > concern >> in >> >>> > > general@ list. >> >>> > > >> >>> > > - Henry >> >>> > > >> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell < >> pwend...@gmail.com >> >>> > >> >>> > wrote: >> >>> > >> Hey Henry, >> >>> > >> >> >>> > >> One thing a lot of people do during the vote is test the >> binaries and >> >>> > >> make sure they work. This is really valuable. If you'd like >> >>> > >> I >> could >> >>> > >> add a caveat to the vote thread explaining that we are only >> voting on >> >>> > >> the source. >> >>> > >> >> >>> > >> - Patrick >> >>> > >> >> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < >> >>> > henry.sapu...@gmail.com> wrote: >> >>> > >>> Actually we should be fine putting the binaries there as >> >>> > >>> long >> as the >> >>> > >>> VOTE is for the source. >> >>> > >>> >> >>> > >>> Let's verify with sebb in the general@ list about his concern. >> >>> > >>> >> >>> > >>> - Henry >> >>> > >>> >> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < >> >>> > henry.sapu...@gmail.com> wrote: >> >>> > >>>> Hi Patrick, as sebb has mentioned let's move the binaries >> >>> > >>>> from >> the >> >>> > >>>> voting directory in your people.apache.org directory. >> >>> > >>>> ASF release voting is for source code and not binaries, >> >>> > >>>> and technically we provide binaries for convenience. &
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi here, Do we have plan to upgrade protobuf from 2.4.1 to 2.5.0? PB has some uncompatable API between these two versions. Hadoop-2.x using protobuf-2.5.0 but if some guys want to run Spark on mesos, then mesos using protobuf-2.4.1 currently. so we may discuss here for a better solution. On Mon, Dec 16, 2013 at 7:42 AM, Azuryy Yu wrote: > Thanks Patrick. > On 16 Dec 2013 02:43, "Patrick Wendell" wrote: > >> You can checkout the docs mentioned in the vote thread. There is also >> a pre-build binary for hadoop2 that is compiled for YARN 2.2 >> >> - Patrick >> >> On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: >> > yarn 2.2, not yarn 0.22, I am so sorry. >> > >> > >> > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: >> > >> >> Hi, >> >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? >> >> Thanks. >> >> >> >> >> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra < >> henry.sapu...@gmail.com>wrote: >> >> >> >>> Yeah seems like it. He was ok with our prev release. >> >>> Let's wait for his reply >> >>> >> >>> On Saturday, December 14, 2013, Patrick Wendell wrote: >> >>> >> >>> > Henry - from that thread it looks like sebb's concern was something >> >>> > different than this. >> >>> > >> >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >> >>> henry.sapu...@gmail.com> >> >>> > wrote: >> >>> > > Hi Patrick, >> >>> > > >> >>> > > Yeap I agree, but technically ASF VOTE release on source only, >> there >> >>> > > even debate about it =), so putting it in the vote staging >> artifact >> >>> > > could confuse people because in our case we do package 3rd party >> >>> > > libraries in the binary jars. >> >>> > > >> >>> > > I have sent email to sebb asking clarification about his concern >> in >> >>> > > general@ list. >> >>> > > >> >>> > > - Henry >> >>> > > >> >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell < >> pwend...@gmail.com >> >>> > >> >>> > wrote: >> >>> > >> Hey Henry, >> >>> > >> >> >>> > >> One thing a lot of people do during the vote is test the >> binaries and >> >>> > >> make sure they work. This is really valuable. If you'd like I >> could >> >>> > >> add a caveat to the vote thread explaining that we are only >> voting on >> >>> > >> the source. >> >>> > >> >> >>> > >> - Patrick >> >>> > >> >> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < >> >>> > henry.sapu...@gmail.com> wrote: >> >>> > >>> Actually we should be fine putting the binaries there as long >> as the >> >>> > >>> VOTE is for the source. >> >>> > >>> >> >>> > >>> Let's verify with sebb in the general@ list about his concern. >> >>> > >>> >> >>> > >>> - Henry >> >>> > >>> >> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < >> >>> > henry.sapu...@gmail.com> wrote: >> >>> > Hi Patrick, as sebb has mentioned let's move the binaries from >> the >> >>> > voting directory in your people.apache.org directory. >> >>> > ASF release voting is for source code and not binaries, and >> >>> > technically we provide binaries for convenience. >> >>> > >> >>> > And add link to the KEYS location in the dist[1] to let verify >> >>> > signatures. >> >>> > >> >>> > Sorry for the late response to the VOTE thread, guys. >> >>> > >> >>> > - Henry >> >>> > >> >>> > [1] >> >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS >> >>> > >> >>> > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell < >> >>> pwend...@gmail.com> >> >>> > wrote: >> >>> > > The vote is now closed. This vote passes with 5 PPMC +1's and >> no 0 >> >>> > or -1 >> >>> > > votes. >> >>> > > >> >>> > > +1 (5 Total) >> >>> > > Matei Zaharia* >> >>> > > Nick Pentreath* >> >>> > > Patrick Wendell* >> >>> > > Prashant Sharma* >> >>> > > Tom Graves* >> >>> > > >> >>> > > 0 (0 Total) >> >>> > > >> >>> > > -1 (0 Total) >> >>> > > >> >>> > > * = Binding Vote >> >>> > > >> >>> > > As per the incubator release guide [1] I'll be sending this >> to the >> >>> > > general incubator list for a final vote from IPMC members. >> >>> > > >> >>> > > [1] >> >>> > > >> >>> > >> >>> >> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- >> >>> > > vote >> >>> > > >> >>> > > >> >>> > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan >> wrote: >> >>> > > >> >>> > >> I'd be personally fine with a standard workflow of >> assemble-deps >> >>> + >> >>> > >> packaging just the Spark files as separate packages, if it >> >>> speeds up >> >>> > >> everyone's development time. >> >>> > >> >> >>> > >> >> >>> > >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < >> >>> > m...@clearstorydata.com >> >>> > >> >wrote: >> >>> > >> >> >>> > >> > I don't know how to make sense of the numbers, but here's >> what >> >>> > I've got >> >>> > >> > from a very small sample size. >> >>> >> >> >> >> >> >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Thanks Patrick. On 16 Dec 2013 02:43, "Patrick Wendell" wrote: > You can checkout the docs mentioned in the vote thread. There is also > a pre-build binary for hadoop2 that is compiled for YARN 2.2 > > - Patrick > > On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: > > yarn 2.2, not yarn 0.22, I am so sorry. > > > > > > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: > > > >> Hi, > >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? > >> Thanks. > >> > >> > >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra >wrote: > >> > >>> Yeah seems like it. He was ok with our prev release. > >>> Let's wait for his reply > >>> > >>> On Saturday, December 14, 2013, Patrick Wendell wrote: > >>> > >>> > Henry - from that thread it looks like sebb's concern was something > >>> > different than this. > >>> > > >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < > >>> henry.sapu...@gmail.com> > >>> > wrote: > >>> > > Hi Patrick, > >>> > > > >>> > > Yeap I agree, but technically ASF VOTE release on source only, > there > >>> > > even debate about it =), so putting it in the vote staging artifact > >>> > > could confuse people because in our case we do package 3rd party > >>> > > libraries in the binary jars. > >>> > > > >>> > > I have sent email to sebb asking clarification about his concern in > >>> > > general@ list. > >>> > > > >>> > > - Henry > >>> > > > >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell < > pwend...@gmail.com > >>> > > >>> > wrote: > >>> > >> Hey Henry, > >>> > >> > >>> > >> One thing a lot of people do during the vote is test the binaries > and > >>> > >> make sure they work. This is really valuable. If you'd like I > could > >>> > >> add a caveat to the vote thread explaining that we are only > voting on > >>> > >> the source. > >>> > >> > >>> > >> - Patrick > >>> > >> > >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < > >>> > henry.sapu...@gmail.com> wrote: > >>> > >>> Actually we should be fine putting the binaries there as long as > the > >>> > >>> VOTE is for the source. > >>> > >>> > >>> > >>> Let's verify with sebb in the general@ list about his concern. > >>> > >>> > >>> > >>> - Henry > >>> > >>> > >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < > >>> > henry.sapu...@gmail.com> wrote: > >>> > Hi Patrick, as sebb has mentioned let's move the binaries from > the > >>> > voting directory in your people.apache.org directory. > >>> > ASF release voting is for source code and not binaries, and > >>> > technically we provide binaries for convenience. > >>> > > >>> > And add link to the KEYS location in the dist[1] to let verify > >>> > signatures. > >>> > > >>> > Sorry for the late response to the VOTE thread, guys. > >>> > > >>> > - Henry > >>> > > >>> > [1] > >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS > >>> > > >>> > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell < > >>> pwend...@gmail.com> > >>> > wrote: > >>> > > The vote is now closed. This vote passes with 5 PPMC +1's and > no 0 > >>> > or -1 > >>> > > votes. > >>> > > > >>> > > +1 (5 Total) > >>> > > Matei Zaharia* > >>> > > Nick Pentreath* > >>> > > Patrick Wendell* > >>> > > Prashant Sharma* > >>> > > Tom Graves* > >>> > > > >>> > > 0 (0 Total) > >>> > > > >>> > > -1 (0 Total) > >>> > > > >>> > > * = Binding Vote > >>> > > > >>> > > As per the incubator release guide [1] I'll be sending this to > the > >>> > > general incubator list for a final vote from IPMC members. > >>> > > > >>> > > [1] > >>> > > > >>> > > >>> > http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- > >>> > > vote > >>> > > > >>> > > > >>> > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan > wrote: > >>> > > > >>> > >> I'd be personally fine with a standard workflow of > assemble-deps > >>> + > >>> > >> packaging just the Spark files as separate packages, if it > >>> speeds up > >>> > >> everyone's development time. > >>> > >> > >>> > >> > >>> > >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < > >>> > m...@clearstorydata.com > >>> > >> >wrote: > >>> > >> > >>> > >> > I don't know how to make sense of the numbers, but here's > what > >>> > I've got > >>> > >> > from a very small sample size. > >>> > >> > >> >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
You can checkout the docs mentioned in the vote thread. There is also a pre-build binary for hadoop2 that is compiled for YARN 2.2 - Patrick On Sun, Dec 15, 2013 at 4:31 AM, Azuryy Yu wrote: > yarn 2.2, not yarn 0.22, I am so sorry. > > > On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: > >> Hi, >> Spark-0.8.1 supports yarn 0.22 right? where to find the release note? >> Thanks. >> >> >> On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra >> wrote: >> >>> Yeah seems like it. He was ok with our prev release. >>> Let's wait for his reply >>> >>> On Saturday, December 14, 2013, Patrick Wendell wrote: >>> >>> > Henry - from that thread it looks like sebb's concern was something >>> > different than this. >>> > >>> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >>> henry.sapu...@gmail.com> >>> > wrote: >>> > > Hi Patrick, >>> > > >>> > > Yeap I agree, but technically ASF VOTE release on source only, there >>> > > even debate about it =), so putting it in the vote staging artifact >>> > > could confuse people because in our case we do package 3rd party >>> > > libraries in the binary jars. >>> > > >>> > > I have sent email to sebb asking clarification about his concern in >>> > > general@ list. >>> > > >>> > > - Henry >>> > > >>> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell >> > >>> > wrote: >>> > >> Hey Henry, >>> > >> >>> > >> One thing a lot of people do during the vote is test the binaries and >>> > >> make sure they work. This is really valuable. If you'd like I could >>> > >> add a caveat to the vote thread explaining that we are only voting on >>> > >> the source. >>> > >> >>> > >> - Patrick >>> > >> >>> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < >>> > henry.sapu...@gmail.com> wrote: >>> > >>> Actually we should be fine putting the binaries there as long as the >>> > >>> VOTE is for the source. >>> > >>> >>> > >>> Let's verify with sebb in the general@ list about his concern. >>> > >>> >>> > >>> - Henry >>> > >>> >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < >>> > henry.sapu...@gmail.com> wrote: >>> > Hi Patrick, as sebb has mentioned let's move the binaries from the >>> > voting directory in your people.apache.org directory. >>> > ASF release voting is for source code and not binaries, and >>> > technically we provide binaries for convenience. >>> > >>> > And add link to the KEYS location in the dist[1] to let verify >>> > signatures. >>> > >>> > Sorry for the late response to the VOTE thread, guys. >>> > >>> > - Henry >>> > >>> > [1] >>> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS >>> > >>> > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell < >>> pwend...@gmail.com> >>> > wrote: >>> > > The vote is now closed. This vote passes with 5 PPMC +1's and no 0 >>> > or -1 >>> > > votes. >>> > > >>> > > +1 (5 Total) >>> > > Matei Zaharia* >>> > > Nick Pentreath* >>> > > Patrick Wendell* >>> > > Prashant Sharma* >>> > > Tom Graves* >>> > > >>> > > 0 (0 Total) >>> > > >>> > > -1 (0 Total) >>> > > >>> > > * = Binding Vote >>> > > >>> > > As per the incubator release guide [1] I'll be sending this to the >>> > > general incubator list for a final vote from IPMC members. >>> > > >>> > > [1] >>> > > >>> > >>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- >>> > > vote >>> > > >>> > > >>> > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: >>> > > >>> > >> I'd be personally fine with a standard workflow of assemble-deps >>> + >>> > >> packaging just the Spark files as separate packages, if it >>> speeds up >>> > >> everyone's development time. >>> > >> >>> > >> >>> > >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < >>> > m...@clearstorydata.com >>> > >> >wrote: >>> > >> >>> > >> > I don't know how to make sense of the numbers, but here's what >>> > I've got >>> > >> > from a very small sample size. >>> >> >>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi, Spark-0.8.1 supports yarn 0.22 right? where to find the release note? Thanks. On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra wrote: > Yeah seems like it. He was ok with our prev release. > Let's wait for his reply > > On Saturday, December 14, 2013, Patrick Wendell wrote: > > > Henry - from that thread it looks like sebb's concern was something > > different than this. > > > > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra > > > wrote: > > > Hi Patrick, > > > > > > Yeap I agree, but technically ASF VOTE release on source only, there > > > even debate about it =), so putting it in the vote staging artifact > > > could confuse people because in our case we do package 3rd party > > > libraries in the binary jars. > > > > > > I have sent email to sebb asking clarification about his concern in > > > general@ list. > > > > > > - Henry > > > > > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell > > wrote: > > >> Hey Henry, > > >> > > >> One thing a lot of people do during the vote is test the binaries and > > >> make sure they work. This is really valuable. If you'd like I could > > >> add a caveat to the vote thread explaining that we are only voting on > > >> the source. > > >> > > >> - Patrick > > >> > > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < > > henry.sapu...@gmail.com> wrote: > > >>> Actually we should be fine putting the binaries there as long as the > > >>> VOTE is for the source. > > >>> > > >>> Let's verify with sebb in the general@ list about his concern. > > >>> > > >>> - Henry > > >>> > > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < > > henry.sapu...@gmail.com> wrote: > > Hi Patrick, as sebb has mentioned let's move the binaries from the > > voting directory in your people.apache.org directory. > > ASF release voting is for source code and not binaries, and > > technically we provide binaries for convenience. > > > > And add link to the KEYS location in the dist[1] to let verify > > signatures. > > > > Sorry for the late response to the VOTE thread, guys. > > > > - Henry > > > > [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS > > > > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell < > pwend...@gmail.com> > > wrote: > > > The vote is now closed. This vote passes with 5 PPMC +1's and no 0 > > or -1 > > > votes. > > > > > > +1 (5 Total) > > > Matei Zaharia* > > > Nick Pentreath* > > > Patrick Wendell* > > > Prashant Sharma* > > > Tom Graves* > > > > > > 0 (0 Total) > > > > > > -1 (0 Total) > > > > > > * = Binding Vote > > > > > > As per the incubator release guide [1] I'll be sending this to the > > > general incubator list for a final vote from IPMC members. > > > > > > [1] > > > > > > http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- > > > vote > > > > > > > > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: > > > > > >> I'd be personally fine with a standard workflow of assemble-deps + > > >> packaging just the Spark files as separate packages, if it speeds > up > > >> everyone's development time. > > >> > > >> > > >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < > > m...@clearstorydata.com > > >> >wrote: > > >> > > >> > I don't know how to make sense of the numbers, but here's what > > I've got > > >> > from a very small sample size. >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
yarn 2.2, not yarn 0.22, I am so sorry. On Sun, Dec 15, 2013 at 8:31 PM, Azuryy Yu wrote: > Hi, > Spark-0.8.1 supports yarn 0.22 right? where to find the release note? > Thanks. > > > On Sun, Dec 15, 2013 at 3:20 AM, Henry Saputra wrote: > >> Yeah seems like it. He was ok with our prev release. >> Let's wait for his reply >> >> On Saturday, December 14, 2013, Patrick Wendell wrote: >> >> > Henry - from that thread it looks like sebb's concern was something >> > different than this. >> > >> > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra < >> henry.sapu...@gmail.com> >> > wrote: >> > > Hi Patrick, >> > > >> > > Yeap I agree, but technically ASF VOTE release on source only, there >> > > even debate about it =), so putting it in the vote staging artifact >> > > could confuse people because in our case we do package 3rd party >> > > libraries in the binary jars. >> > > >> > > I have sent email to sebb asking clarification about his concern in >> > > general@ list. >> > > >> > > - Henry >> > > >> > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell > > >> > wrote: >> > >> Hey Henry, >> > >> >> > >> One thing a lot of people do during the vote is test the binaries and >> > >> make sure they work. This is really valuable. If you'd like I could >> > >> add a caveat to the vote thread explaining that we are only voting on >> > >> the source. >> > >> >> > >> - Patrick >> > >> >> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < >> > henry.sapu...@gmail.com> wrote: >> > >>> Actually we should be fine putting the binaries there as long as the >> > >>> VOTE is for the source. >> > >>> >> > >>> Let's verify with sebb in the general@ list about his concern. >> > >>> >> > >>> - Henry >> > >>> >> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < >> > henry.sapu...@gmail.com> wrote: >> > Hi Patrick, as sebb has mentioned let's move the binaries from the >> > voting directory in your people.apache.org directory. >> > ASF release voting is for source code and not binaries, and >> > technically we provide binaries for convenience. >> > >> > And add link to the KEYS location in the dist[1] to let verify >> > signatures. >> > >> > Sorry for the late response to the VOTE thread, guys. >> > >> > - Henry >> > >> > [1] >> https://dist.apache.org/repos/dist/release/incubator/spark/KEYS >> > >> > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell < >> pwend...@gmail.com> >> > wrote: >> > > The vote is now closed. This vote passes with 5 PPMC +1's and no 0 >> > or -1 >> > > votes. >> > > >> > > +1 (5 Total) >> > > Matei Zaharia* >> > > Nick Pentreath* >> > > Patrick Wendell* >> > > Prashant Sharma* >> > > Tom Graves* >> > > >> > > 0 (0 Total) >> > > >> > > -1 (0 Total) >> > > >> > > * = Binding Vote >> > > >> > > As per the incubator release guide [1] I'll be sending this to the >> > > general incubator list for a final vote from IPMC members. >> > > >> > > [1] >> > > >> > >> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- >> > > vote >> > > >> > > >> > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: >> > > >> > >> I'd be personally fine with a standard workflow of assemble-deps >> + >> > >> packaging just the Spark files as separate packages, if it >> speeds up >> > >> everyone's development time. >> > >> >> > >> >> > >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < >> > m...@clearstorydata.com >> > >> >wrote: >> > >> >> > >> > I don't know how to make sense of the numbers, but here's what >> > I've got >> > >> > from a very small sample size. >> > >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Yeah seems like it. He was ok with our prev release. Let's wait for his reply On Saturday, December 14, 2013, Patrick Wendell wrote: > Henry - from that thread it looks like sebb's concern was something > different than this. > > On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra > wrote: > > Hi Patrick, > > > > Yeap I agree, but technically ASF VOTE release on source only, there > > even debate about it =), so putting it in the vote staging artifact > > could confuse people because in our case we do package 3rd party > > libraries in the binary jars. > > > > I have sent email to sebb asking clarification about his concern in > > general@ list. > > > > - Henry > > > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell > wrote: > >> Hey Henry, > >> > >> One thing a lot of people do during the vote is test the binaries and > >> make sure they work. This is really valuable. If you'd like I could > >> add a caveat to the vote thread explaining that we are only voting on > >> the source. > >> > >> - Patrick > >> > >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra < > henry.sapu...@gmail.com> wrote: > >>> Actually we should be fine putting the binaries there as long as the > >>> VOTE is for the source. > >>> > >>> Let's verify with sebb in the general@ list about his concern. > >>> > >>> - Henry > >>> > >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra < > henry.sapu...@gmail.com> wrote: > Hi Patrick, as sebb has mentioned let's move the binaries from the > voting directory in your people.apache.org directory. > ASF release voting is for source code and not binaries, and > technically we provide binaries for convenience. > > And add link to the KEYS location in the dist[1] to let verify > signatures. > > Sorry for the late response to the VOTE thread, guys. > > - Henry > > [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS > > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell > wrote: > > The vote is now closed. This vote passes with 5 PPMC +1's and no 0 > or -1 > > votes. > > > > +1 (5 Total) > > Matei Zaharia* > > Nick Pentreath* > > Patrick Wendell* > > Prashant Sharma* > > Tom Graves* > > > > 0 (0 Total) > > > > -1 (0 Total) > > > > * = Binding Vote > > > > As per the incubator release guide [1] I'll be sending this to the > > general incubator list for a final vote from IPMC members. > > > > [1] > > > http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- > > vote > > > > > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: > > > >> I'd be personally fine with a standard workflow of assemble-deps + > >> packaging just the Spark files as separate packages, if it speeds up > >> everyone's development time. > >> > >> > >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra < > m...@clearstorydata.com > >> >wrote: > >> > >> > I don't know how to make sense of the numbers, but here's what > I've got > >> > from a very small sample size.
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Henry - from that thread it looks like sebb's concern was something different than this. On Sat, Dec 14, 2013 at 11:08 AM, Henry Saputra wrote: > Hi Patrick, > > Yeap I agree, but technically ASF VOTE release on source only, there > even debate about it =), so putting it in the vote staging artifact > could confuse people because in our case we do package 3rd party > libraries in the binary jars. > > I have sent email to sebb asking clarification about his concern in > general@ list. > > - Henry > > On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell wrote: >> Hey Henry, >> >> One thing a lot of people do during the vote is test the binaries and >> make sure they work. This is really valuable. If you'd like I could >> add a caveat to the vote thread explaining that we are only voting on >> the source. >> >> - Patrick >> >> On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra >> wrote: >>> Actually we should be fine putting the binaries there as long as the >>> VOTE is for the source. >>> >>> Let's verify with sebb in the general@ list about his concern. >>> >>> - Henry >>> >>> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra >>> wrote: Hi Patrick, as sebb has mentioned let's move the binaries from the voting directory in your people.apache.org directory. ASF release voting is for source code and not binaries, and technically we provide binaries for convenience. And add link to the KEYS location in the dist[1] to let verify signatures. Sorry for the late response to the VOTE thread, guys. - Henry [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell wrote: > The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 > votes. > > +1 (5 Total) > Matei Zaharia* > Nick Pentreath* > Patrick Wendell* > Prashant Sharma* > Tom Graves* > > 0 (0 Total) > > -1 (0 Total) > > * = Binding Vote > > As per the incubator release guide [1] I'll be sending this to the > general incubator list for a final vote from IPMC members. > > [1] > http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- > vote > > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: > >> I'd be personally fine with a standard workflow of assemble-deps + >> packaging just the Spark files as separate packages, if it speeds up >> everyone's development time. >> >> >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra > >wrote: >> >> > I don't know how to make sense of the numbers, but here's what I've got >> > from a very small sample size. >> > >> > For both v0.8.0-incubating and v0.8.1-incubating, building separate >> > assemblies is faster than `./sbt/sbt assembly` and the times for >> > building >> > separate assemblies for 0.8.0 and 0.8.1 are about the same. >> > >> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as >> the >> > sum of the separate assemblies. >> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as >> the >> > sum of the separate assemblies. >> > >> > Weird. >> > >> > >> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell > > >wrote: >> > >> > > I'll +1 myself also. >> > > >> > > For anyone who has the slow build problem: does this issue happen >> > > when >> > > building v0.8.0-incubating also? Trying to figure out whether it's >> > > related to something we added in 0.8.1 or if it's a long standing >> > > issue. >> > > >> > > - Patrick >> > > >> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < >> matei.zaha...@gmail.com >> > > >> > > wrote: >> > > > Woah, weird, but definitely good to know. >> > > > >> > > > If you’re doing Spark development, there’s also a more convenient >> > option >> > > added by Shivaram in the master branch. You can do sbt assemble-deps >> > > to >> > > package *just* the dependencies of each project in a special assembly >> > JAR, >> > > and then use sbt compile to update the code. This will use the >> > > classes >> > > directly out of the target/scala-2.9.3/classes directories. You have >> > > to >> > > redo assemble-deps only if your external dependencies change. >> > > > >> > > > Matei >> > > > >> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma >> > > wrote: >> > > > >> > > >> I hope this PR >> > > >> https://github.com/apache/incubator-spark/pull/252can >> > > help. >> > > >> Again this is not a blocker for the release from my side either. >> > > >> >> > > >> >> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < >> > m...@clearstorydata.com >> > > >wrote: >> > > >> >> > > >>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi Patrick, Yeap I agree, but technically ASF VOTE release on source only, there even debate about it =), so putting it in the vote staging artifact could confuse people because in our case we do package 3rd party libraries in the binary jars. I have sent email to sebb asking clarification about his concern in general@ list. - Henry On Sat, Dec 14, 2013 at 10:56 AM, Patrick Wendell wrote: > Hey Henry, > > One thing a lot of people do during the vote is test the binaries and > make sure they work. This is really valuable. If you'd like I could > add a caveat to the vote thread explaining that we are only voting on > the source. > > - Patrick > > On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra > wrote: >> Actually we should be fine putting the binaries there as long as the >> VOTE is for the source. >> >> Let's verify with sebb in the general@ list about his concern. >> >> - Henry >> >> On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra >> wrote: >>> Hi Patrick, as sebb has mentioned let's move the binaries from the >>> voting directory in your people.apache.org directory. >>> ASF release voting is for source code and not binaries, and >>> technically we provide binaries for convenience. >>> >>> And add link to the KEYS location in the dist[1] to let verify signatures. >>> >>> Sorry for the late response to the VOTE thread, guys. >>> >>> - Henry >>> >>> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS >>> >>> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell wrote: The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 votes. +1 (5 Total) Matei Zaharia* Nick Pentreath* Patrick Wendell* Prashant Sharma* Tom Graves* 0 (0 Total) -1 (0 Total) * = Binding Vote As per the incubator release guide [1] I'll be sending this to the general incubator list for a final vote from IPMC members. [1] http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- vote On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: > I'd be personally fine with a standard workflow of assemble-deps + > packaging just the Spark files as separate packages, if it speeds up > everyone's development time. > > > On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra >wrote: > > > I don't know how to make sense of the numbers, but here's what I've got > > from a very small sample size. > > > > For both v0.8.0-incubating and v0.8.1-incubating, building separate > > assemblies is faster than `./sbt/sbt assembly` and the times for > > building > > separate assemblies for 0.8.0 and 0.8.1 are about the same. > > > > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as > the > > sum of the separate assemblies. > > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as > the > > sum of the separate assemblies. > > > > Weird. > > > > > > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell > >wrote: > > > > > I'll +1 myself also. > > > > > > For anyone who has the slow build problem: does this issue happen when > > > building v0.8.0-incubating also? Trying to figure out whether it's > > > related to something we added in 0.8.1 or if it's a long standing > > > issue. > > > > > > - Patrick > > > > > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < > matei.zaha...@gmail.com > > > > > > wrote: > > > > Woah, weird, but definitely good to know. > > > > > > > > If you’re doing Spark development, there’s also a more convenient > > option > > > added by Shivaram in the master branch. You can do sbt assemble-deps > > > to > > > package *just* the dependencies of each project in a special assembly > > JAR, > > > and then use sbt compile to update the code. This will use the classes > > > directly out of the target/scala-2.9.3/classes directories. You have > > > to > > > redo assemble-deps only if your external dependencies change. > > > > > > > > Matei > > > > > > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > > > wrote: > > > > > > > >> I hope this PR > > > >> https://github.com/apache/incubator-spark/pull/252can > > > help. > > > >> Again this is not a blocker for the release from my side either. > > > >> > > > >> > > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < > > m...@clearstorydata.com > > > >wrote: > > > >> > > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt > assembly` > > > takes > > > >>> a long, long, long time to complete (a MBP, in my case), > building > > > three > > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > > > >>> examples/assembly`, `./sbt/sbt tools/asse
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hey Henry, One thing a lot of people do during the vote is test the binaries and make sure they work. This is really valuable. If you'd like I could add a caveat to the vote thread explaining that we are only voting on the source. - Patrick On Sat, Dec 14, 2013 at 10:40 AM, Henry Saputra wrote: > Actually we should be fine putting the binaries there as long as the > VOTE is for the source. > > Let's verify with sebb in the general@ list about his concern. > > - Henry > > On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra > wrote: >> Hi Patrick, as sebb has mentioned let's move the binaries from the >> voting directory in your people.apache.org directory. >> ASF release voting is for source code and not binaries, and >> technically we provide binaries for convenience. >> >> And add link to the KEYS location in the dist[1] to let verify signatures. >> >> Sorry for the late response to the VOTE thread, guys. >> >> - Henry >> >> [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS >> >> On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell wrote: >>> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 >>> votes. >>> >>> +1 (5 Total) >>> Matei Zaharia* >>> Nick Pentreath* >>> Patrick Wendell* >>> Prashant Sharma* >>> Tom Graves* >>> >>> 0 (0 Total) >>> >>> -1 (0 Total) >>> >>> * = Binding Vote >>> >>> As per the incubator release guide [1] I'll be sending this to the >>> general incubator list for a final vote from IPMC members. >>> >>> [1] >>> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- >>> vote >>> >>> >>> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: >>> I'd be personally fine with a standard workflow of assemble-deps + packaging just the Spark files as separate packages, if it speeds up everyone's development time. On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra >>> >wrote: > I don't know how to make sense of the numbers, but here's what I've got > from a very small sample size. > > For both v0.8.0-incubating and v0.8.1-incubating, building separate > assemblies is faster than `./sbt/sbt assembly` and the times for building > separate assemblies for 0.8.0 and 0.8.1 are about the same. > > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the > sum of the separate assemblies. > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the > sum of the separate assemblies. > > Weird. > > > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell >>> > >wrote: > > > I'll +1 myself also. > > > > For anyone who has the slow build problem: does this issue happen when > > building v0.8.0-incubating also? Trying to figure out whether it's > > related to something we added in 0.8.1 or if it's a long standing > > issue. > > > > - Patrick > > > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < matei.zaha...@gmail.com > > > > wrote: > > > Woah, weird, but definitely good to know. > > > > > > If you’re doing Spark development, there’s also a more convenient > option > > added by Shivaram in the master branch. You can do sbt assemble-deps to > > package *just* the dependencies of each project in a special assembly > JAR, > > and then use sbt compile to update the code. This will use the classes > > directly out of the target/scala-2.9.3/classes directories. You have to > > redo assemble-deps only if your external dependencies change. > > > > > > Matei > > > > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > > wrote: > > > > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can > > help. > > >> Again this is not a blocker for the release from my side either. > > >> > > >> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < > m...@clearstorydata.com > > >wrote: > > >> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` > > takes > > >>> a long, long, long time to complete (a MBP, in my case), building > > three > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less > > time. > > >>> > > >>> > > >>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < > > scrapco...@gmail.com > > wrote: > > >>> > > forgot to mention, after running sbt/sbt assembly/assembly running > > >>> sbt/sbt > > examples/assembly takes just 37s. Not to mention my hardware is not > > >>> really > > great. > > > > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < > > scrapco...@gmail.com > > >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Actually we should be fine putting the binaries there as long as the VOTE is for the source. Let's verify with sebb in the general@ list about his concern. - Henry On Sat, Dec 14, 2013 at 10:31 AM, Henry Saputra wrote: > Hi Patrick, as sebb has mentioned let's move the binaries from the > voting directory in your people.apache.org directory. > ASF release voting is for source code and not binaries, and > technically we provide binaries for convenience. > > And add link to the KEYS location in the dist[1] to let verify signatures. > > Sorry for the late response to the VOTE thread, guys. > > - Henry > > [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS > > On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell wrote: >> The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 >> votes. >> >> +1 (5 Total) >> Matei Zaharia* >> Nick Pentreath* >> Patrick Wendell* >> Prashant Sharma* >> Tom Graves* >> >> 0 (0 Total) >> >> -1 (0 Total) >> >> * = Binding Vote >> >> As per the incubator release guide [1] I'll be sending this to the >> general incubator list for a final vote from IPMC members. >> >> [1] >> http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- >> vote >> >> >> On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: >> >>> I'd be personally fine with a standard workflow of assemble-deps + >>> packaging just the Spark files as separate packages, if it speeds up >>> everyone's development time. >>> >>> >>> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra >> >wrote: >>> >>> > I don't know how to make sense of the numbers, but here's what I've got >>> > from a very small sample size. >>> > >>> > For both v0.8.0-incubating and v0.8.1-incubating, building separate >>> > assemblies is faster than `./sbt/sbt assembly` and the times for building >>> > separate assemblies for 0.8.0 and 0.8.1 are about the same. >>> > >>> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as >>> the >>> > sum of the separate assemblies. >>> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as >>> the >>> > sum of the separate assemblies. >>> > >>> > Weird. >>> > >>> > >>> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell >> > >wrote: >>> > >>> > > I'll +1 myself also. >>> > > >>> > > For anyone who has the slow build problem: does this issue happen when >>> > > building v0.8.0-incubating also? Trying to figure out whether it's >>> > > related to something we added in 0.8.1 or if it's a long standing >>> > > issue. >>> > > >>> > > - Patrick >>> > > >>> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < >>> matei.zaha...@gmail.com >>> > > >>> > > wrote: >>> > > > Woah, weird, but definitely good to know. >>> > > > >>> > > > If you’re doing Spark development, there’s also a more convenient >>> > option >>> > > added by Shivaram in the master branch. You can do sbt assemble-deps to >>> > > package *just* the dependencies of each project in a special assembly >>> > JAR, >>> > > and then use sbt compile to update the code. This will use the classes >>> > > directly out of the target/scala-2.9.3/classes directories. You have to >>> > > redo assemble-deps only if your external dependencies change. >>> > > > >>> > > > Matei >>> > > > >>> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma >>> > > wrote: >>> > > > >>> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can >>> > > help. >>> > > >> Again this is not a blocker for the release from my side either. >>> > > >> >>> > > >> >>> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < >>> > m...@clearstorydata.com >>> > > >wrote: >>> > > >> >>> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt >>> assembly` >>> > > takes >>> > > >>> a long, long, long time to complete (a MBP, in my case), >>> building >>> > > three >>> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >>> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much >>> less >>> > > time. >>> > > >>> >>> > > >>> >>> > > >>> >>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < >>> > > scrapco...@gmail.com >>> > > wrote: >>> > > >>> >>> > > forgot to mention, after running sbt/sbt assembly/assembly running >>> > > >>> sbt/sbt >>> > > examples/assembly takes just 37s. Not to mention my hardware is >>> not >>> > > >>> really >>> > > great. >>> > > >>> > > >>> > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < >>> > > scrapco...@gmail.com >>> > > > wrote: >>> > > >>> > > > Hi Patrick and Matei, >>> > > > >>> > > > Was trying out this and followed the quick start guide which says >>> > do >>> > > > sbt/sbt assembly, like few others I was also stuck for few >>> minutes >>> > on >>> > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is >>> > > much >>> > > > faster. >>> > > > >>> > > > Should we change the documentation to reflect this. It
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi Patrick, as sebb has mentioned let's move the binaries from the voting directory in your people.apache.org directory. ASF release voting is for source code and not binaries, and technically we provide binaries for convenience. And add link to the KEYS location in the dist[1] to let verify signatures. Sorry for the late response to the VOTE thread, guys. - Henry [1] https://dist.apache.org/repos/dist/release/incubator/spark/KEYS On Fri, Dec 13, 2013 at 6:37 PM, Patrick Wendell wrote: > The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 > votes. > > +1 (5 Total) > Matei Zaharia* > Nick Pentreath* > Patrick Wendell* > Prashant Sharma* > Tom Graves* > > 0 (0 Total) > > -1 (0 Total) > > * = Binding Vote > > As per the incubator release guide [1] I'll be sending this to the > general incubator list for a final vote from IPMC members. > > [1] > http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- > vote > > > On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: > >> I'd be personally fine with a standard workflow of assemble-deps + >> packaging just the Spark files as separate packages, if it speeds up >> everyone's development time. >> >> >> On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra > >wrote: >> >> > I don't know how to make sense of the numbers, but here's what I've got >> > from a very small sample size. >> > >> > For both v0.8.0-incubating and v0.8.1-incubating, building separate >> > assemblies is faster than `./sbt/sbt assembly` and the times for building >> > separate assemblies for 0.8.0 and 0.8.1 are about the same. >> > >> > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as >> the >> > sum of the separate assemblies. >> > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as >> the >> > sum of the separate assemblies. >> > >> > Weird. >> > >> > >> > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell > > >wrote: >> > >> > > I'll +1 myself also. >> > > >> > > For anyone who has the slow build problem: does this issue happen when >> > > building v0.8.0-incubating also? Trying to figure out whether it's >> > > related to something we added in 0.8.1 or if it's a long standing >> > > issue. >> > > >> > > - Patrick >> > > >> > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < >> matei.zaha...@gmail.com >> > > >> > > wrote: >> > > > Woah, weird, but definitely good to know. >> > > > >> > > > If you’re doing Spark development, there’s also a more convenient >> > option >> > > added by Shivaram in the master branch. You can do sbt assemble-deps to >> > > package *just* the dependencies of each project in a special assembly >> > JAR, >> > > and then use sbt compile to update the code. This will use the classes >> > > directly out of the target/scala-2.9.3/classes directories. You have to >> > > redo assemble-deps only if your external dependencies change. >> > > > >> > > > Matei >> > > > >> > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma >> > > wrote: >> > > > >> > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can >> > > help. >> > > >> Again this is not a blocker for the release from my side either. >> > > >> >> > > >> >> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < >> > m...@clearstorydata.com >> > > >wrote: >> > > >> >> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt >> assembly` >> > > takes >> > > >>> a long, long, long time to complete (a MBP, in my case), >> building >> > > three >> > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >> > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much >> less >> > > time. >> > > >>> >> > > >>> >> > > >>> >> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < >> > > scrapco...@gmail.com >> > > wrote: >> > > >>> >> > > forgot to mention, after running sbt/sbt assembly/assembly running >> > > >>> sbt/sbt >> > > examples/assembly takes just 37s. Not to mention my hardware is >> not >> > > >>> really >> > > great. >> > > >> > > >> > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < >> > > scrapco...@gmail.com >> > > > wrote: >> > > >> > > > Hi Patrick and Matei, >> > > > >> > > > Was trying out this and followed the quick start guide which says >> > do >> > > > sbt/sbt assembly, like few others I was also stuck for few >> minutes >> > on >> > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is >> > > much >> > > > faster. >> > > > >> > > > Should we change the documentation to reflect this. It will not >> be >> > > >>> great >> > > > for first time users to get stuck there. >> > > > >> > > > >> > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >> > > >>> matei.zaha...@gmail.com >> > > > wrote: >> > > > >> > > >> +1 >> > > >> >> > > >> Built and tested it on Mac OS X. >> > > >> >> > > >> Matei >> > > >> >> > > >>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
The vote is now closed. This vote passes with 5 PPMC +1's and no 0 or -1 votes. +1 (5 Total) Matei Zaharia* Nick Pentreath* Patrick Wendell* Prashant Sharma* Tom Graves* 0 (0 Total) -1 (0 Total) * = Binding Vote As per the incubator release guide [1] I'll be sending this to the general incubator list for a final vote from IPMC members. [1] http://incubator.apache.org/guides/releasemanagement.html#best-practice-incubator-release- vote On Thu, Dec 12, 2013 at 8:59 AM, Evan Chan wrote: > I'd be personally fine with a standard workflow of assemble-deps + > packaging just the Spark files as separate packages, if it speeds up > everyone's development time. > > > On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra >wrote: > > > I don't know how to make sense of the numbers, but here's what I've got > > from a very small sample size. > > > > For both v0.8.0-incubating and v0.8.1-incubating, building separate > > assemblies is faster than `./sbt/sbt assembly` and the times for building > > separate assemblies for 0.8.0 and 0.8.1 are about the same. > > > > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as > the > > sum of the separate assemblies. > > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as > the > > sum of the separate assemblies. > > > > Weird. > > > > > > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell > >wrote: > > > > > I'll +1 myself also. > > > > > > For anyone who has the slow build problem: does this issue happen when > > > building v0.8.0-incubating also? Trying to figure out whether it's > > > related to something we added in 0.8.1 or if it's a long standing > > > issue. > > > > > > - Patrick > > > > > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < > matei.zaha...@gmail.com > > > > > > wrote: > > > > Woah, weird, but definitely good to know. > > > > > > > > If you’re doing Spark development, there’s also a more convenient > > option > > > added by Shivaram in the master branch. You can do sbt assemble-deps to > > > package *just* the dependencies of each project in a special assembly > > JAR, > > > and then use sbt compile to update the code. This will use the classes > > > directly out of the target/scala-2.9.3/classes directories. You have to > > > redo assemble-deps only if your external dependencies change. > > > > > > > > Matei > > > > > > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > > > wrote: > > > > > > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252can > > > help. > > > >> Again this is not a blocker for the release from my side either. > > > >> > > > >> > > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < > > m...@clearstorydata.com > > > >wrote: > > > >> > > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt > assembly` > > > takes > > > >>> a long, long, long time to complete (a MBP, in my case), > building > > > three > > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much > less > > > time. > > > >>> > > > >>> > > > >>> > > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < > > > scrapco...@gmail.com > > > wrote: > > > >>> > > > forgot to mention, after running sbt/sbt assembly/assembly running > > > >>> sbt/sbt > > > examples/assembly takes just 37s. Not to mention my hardware is > not > > > >>> really > > > great. > > > > > > > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < > > > scrapco...@gmail.com > > > > wrote: > > > > > > > Hi Patrick and Matei, > > > > > > > > Was trying out this and followed the quick start guide which says > > do > > > > sbt/sbt assembly, like few others I was also stuck for few > minutes > > on > > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is > > > much > > > > faster. > > > > > > > > Should we change the documentation to reflect this. It will not > be > > > >>> great > > > > for first time users to get stuck there. > > > > > > > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > > > >>> matei.zaha...@gmail.com > > > > wrote: > > > > > > > >> +1 > > > >> > > > >> Built and tested it on Mac OS X. > > > >> > > > >> Matei > > > >> > > > >> > > > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell < > pwend...@gmail.com> > > > wrote: > > > >> > > > >>> Please vote on releasing the following candidate as Apache > Spark > > > >>> (incubating) version 0.8.1. > > > >>> > > > >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): > > > >>> > > > >> > > > > > > >>> > > > > > > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > > >>> > > > >>> The release files, including signatures, digests, etc can be > > found > > > >>> at: > > > >>> http://peop
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
I'd be personally fine with a standard workflow of assemble-deps + packaging just the Spark files as separate packages, if it speeds up everyone's development time. On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra wrote: > I don't know how to make sense of the numbers, but here's what I've got > from a very small sample size. > > For both v0.8.0-incubating and v0.8.1-incubating, building separate > assemblies is faster than `./sbt/sbt assembly` and the times for building > separate assemblies for 0.8.0 and 0.8.1 are about the same. > > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the > sum of the separate assemblies. > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the > sum of the separate assemblies. > > Weird. > > > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell >wrote: > > > I'll +1 myself also. > > > > For anyone who has the slow build problem: does this issue happen when > > building v0.8.0-incubating also? Trying to figure out whether it's > > related to something we added in 0.8.1 or if it's a long standing > > issue. > > > > - Patrick > > > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia > > > wrote: > > > Woah, weird, but definitely good to know. > > > > > > If you’re doing Spark development, there’s also a more convenient > option > > added by Shivaram in the master branch. You can do sbt assemble-deps to > > package *just* the dependencies of each project in a special assembly > JAR, > > and then use sbt compile to update the code. This will use the classes > > directly out of the target/scala-2.9.3/classes directories. You have to > > redo assemble-deps only if your external dependencies change. > > > > > > Matei > > > > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > > wrote: > > > > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can > > help. > > >> Again this is not a blocker for the release from my side either. > > >> > > >> > > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < > m...@clearstorydata.com > > >wrote: > > >> > > >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` > > takes > > >>> a long, long, long time to complete (a MBP, in my case), building > > three > > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less > > time. > > >>> > > >>> > > >>> > > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < > > scrapco...@gmail.com > > wrote: > > >>> > > forgot to mention, after running sbt/sbt assembly/assembly running > > >>> sbt/sbt > > examples/assembly takes just 37s. Not to mention my hardware is not > > >>> really > > great. > > > > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < > > scrapco...@gmail.com > > > wrote: > > > > > Hi Patrick and Matei, > > > > > > Was trying out this and followed the quick start guide which says > do > > > sbt/sbt assembly, like few others I was also stuck for few minutes > on > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is > > much > > > faster. > > > > > > Should we change the documentation to reflect this. It will not be > > >>> great > > > for first time users to get stuck there. > > > > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > > >>> matei.zaha...@gmail.com > > > wrote: > > > > > >> +1 > > >> > > >> Built and tested it on Mac OS X. > > >> > > >> Matei > > >> > > >> > > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell > > wrote: > > >> > > >>> Please vote on releasing the following candidate as Apache Spark > > >>> (incubating) version 0.8.1. > > >>> > > >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): > > >>> > > >> > > > > >>> > > > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > >>> > > >>> The release files, including signatures, digests, etc can be > found > > >>> at: > > >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > > >>> > > >>> Release artifacts are signed with the following key: > > >>> https://people.apache.org/keys/committer/pwendell.asc > > >>> > > >>> The staging repository for this release can be found at: > > >>> > > > > https://repository.apache.org/content/repositories/orgapachespark-040/ > > >>> > > >>> The documentation corresponding to this release can be found at: > > >>> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > > >>> > > >>> For information about the contents of this release see: > > >>> > > >> > > > > >>> > > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > > >>> > > >>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
+1. Built spark on yarn for both hadoop 0.23 and hadoop 2.2.0 on redhat linux using maven. Ran some tests on both a secure Hadoop 0.23 cluster and a secure Hadoop 2.2.0 cluster. Verified signatures and md5. Tom On Tuesday, December 10, 2013 6:49 PM, Patrick Wendell wrote: Please vote on releasing the following candidate as Apache Spark (incubating) version 0.8.1. The tag to be voted on is v0.8.1-incubating (commit b87d31d): https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e The release files, including signatures, digests, etc can be found at: http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ Release artifacts are signed with the following key: https://people.apache.org/keys/committer/pwendell.asc The staging repository for this release can be found at: https://repository.apache.org/content/repositories/orgapachespark-040/ The documentation corresponding to this release can be found at: http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ For information about the contents of this release see: https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 Please vote on releasing this package as Apache Spark 0.8.1-incubating! The vote is open until Saturday, December 14th at 01:00 UTC and passes if a majority of at least 3 +1 PPMC votes are cast. [ ] +1 Release this package as Apache Spark 0.8.1-incubating [ ] -1 Do not release this package because ... To learn more about Apache Spark, please see http://spark.incubator.apache.org/
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
I re-downloaded the source tarball and it works now. Tom On Wednesday, December 11, 2013 6:27 PM, Patrick Wendell wrote: I also talked to a few people who got corrupted binaries when downloading from the people.apache HTTP. In that case the checksum failed but if they re-downloaded it worked. So maybe just re-download and try again? On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell wrote: > Hey Tom, > > I re-verified the signatures and got someone else to do it. It seemed > fine. Here is what I did. > > gpg --recv-key 9E4FE3AF > wget > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc > wget > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz > gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz > gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID 9E4FE3AF > gpg: Good signature from "Patrick Wendell " > > On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra wrote: >> I don't know how to make sense of the numbers, but here's what I've got >> from a very small sample size. >> >> For both v0.8.0-incubating and v0.8.1-incubating, building separate >> assemblies is faster than `./sbt/sbt assembly` and the times for building >> separate assemblies for 0.8.0 and 0.8.1 are about the same. >> >> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the >> sum of the separate assemblies. >> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the >> sum of the separate assemblies. >> >> Weird. >> >> >> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell wrote: >> >>> I'll +1 myself also. >>> >>> For anyone who has the slow build problem: does this issue happen when >>> building v0.8.0-incubating also? Trying to figure out whether it's >>> related to something we added in 0.8.1 or if it's a long standing >>> issue. >>> >>> - Patrick >>> >>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia >>> wrote: >>> > Woah, weird, but definitely good to know. >>> > >>> > If you’re doing Spark development, there’s also a more convenient option >>> added by Shivaram in the master branch. You can do sbt assemble-deps to >>> package *just* the dependencies of each project in a special assembly JAR, >>> and then use sbt compile to update the code. This will use the classes >>> directly out of the target/scala-2.9.3/classes directories. You have to >>> redo assemble-deps only if your external dependencies change. >>> > >>> > Matei >>> > >>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma >>> wrote: >>> > >>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can >>> help. >>> >> Again this is not a blocker for the release from my side either. >>> >> >>> >> >>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra >> >wrote: >>> >> >>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` >>> takes >>> >>> a long, long, long time to complete (a MBP, in my case), building >>> three >>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less >>> time. >>> >>> >>> >>> >>> >>> >>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < >>> scrapco...@gmail.com >>> wrote: >>> >>> >>> forgot to mention, after running sbt/sbt assembly/assembly running >>> >>> sbt/sbt >>> examples/assembly takes just 37s. Not to mention my hardware is not >>> >>> really >>> great. >>> >>> >>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < >>> scrapco...@gmail.com >>> > wrote: >>> >>> > Hi Patrick and Matei, >>> > >>> > Was trying out this and followed the quick start guide which says do >>> > sbt/sbt assembly, like few others I was also stuck for few minutes on >>> > linux. On the other hand if I use sbt/sbt assembly/assembly it is >>> much >>> > faster. >>> > >>> > Should we change the documentation to reflect this. It will not be >>> >>> great >>> > for first time users to get stuck there. >>> > >>> > >>> > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >>> >>> matei.zaha...@gmail.com >>> > wrote: >>> > >>> >> +1 >>> >> >>> >> Built and tested it on Mac OS X. >>> >> >>> >> Matei >>> >> >>> >> >>> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell >>> wrote: >>> >> >>> >>> Please vote on releasing the following candidate as Apache Spark >>> >>> (incubating) version 0.8.1. >>> >>> >>> >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): >>> >>> >>> >> >>> >>> >>> >>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >>> >>> >>> >>> The release files, including signatures, digests, etc can be found >>> >>> at: >>> >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >>> >>> >>> >>> R
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
+1, built and tested on linux. On Thu, Dec 12, 2013 at 5:57 AM, Patrick Wendell wrote: > I also talked to a few people who got corrupted binaries when > downloading from the people.apache HTTP. In that case the checksum > failed but if they re-downloaded it worked. So maybe just re-download > and try again? > > On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell > wrote: > > Hey Tom, > > > > I re-verified the signatures and got someone else to do it. It seemed > > fine. Here is what I did. > > > > gpg --recv-key 9E4FE3AF > > wget > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc > > wget > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz > > gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz > > gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID > 9E4FE3AF > > gpg: Good signature from "Patrick Wendell " > > > > On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra > wrote: > >> I don't know how to make sense of the numbers, but here's what I've got > >> from a very small sample size. > >> > >> For both v0.8.0-incubating and v0.8.1-incubating, building separate > >> assemblies is faster than `./sbt/sbt assembly` and the times for > building > >> separate assemblies for 0.8.0 and 0.8.1 are about the same. > >> > >> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as > the > >> sum of the separate assemblies. > >> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as > the > >> sum of the separate assemblies. > >> > >> Weird. > >> > >> > >> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell >wrote: > >> > >>> I'll +1 myself also. > >>> > >>> For anyone who has the slow build problem: does this issue happen when > >>> building v0.8.0-incubating also? Trying to figure out whether it's > >>> related to something we added in 0.8.1 or if it's a long standing > >>> issue. > >>> > >>> - Patrick > >>> > >>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia < > matei.zaha...@gmail.com> > >>> wrote: > >>> > Woah, weird, but definitely good to know. > >>> > > >>> > If you’re doing Spark development, there’s also a more convenient > option > >>> added by Shivaram in the master branch. You can do sbt assemble-deps to > >>> package *just* the dependencies of each project in a special assembly > JAR, > >>> and then use sbt compile to update the code. This will use the classes > >>> directly out of the target/scala-2.9.3/classes directories. You have to > >>> redo assemble-deps only if your external dependencies change. > >>> > > >>> > Matei > >>> > > >>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > >>> wrote: > >>> > > >>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252can > >>> help. > >>> >> Again this is not a blocker for the release from my side either. > >>> >> > >>> >> > >>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra < > m...@clearstorydata.com > >>> >wrote: > >>> >> > >>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt > assembly` > >>> takes > >>> >>> a long, long, long time to complete (a MBP, in my case), > building > >>> three > >>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > >>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much > less > >>> time. > >>> >>> > >>> >>> > >>> >>> > >>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < > >>> scrapco...@gmail.com > >>> wrote: > >>> >>> > >>> forgot to mention, after running sbt/sbt assembly/assembly running > >>> >>> sbt/sbt > >>> examples/assembly takes just 37s. Not to mention my hardware is > not > >>> >>> really > >>> great. > >>> > >>> > >>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < > >>> scrapco...@gmail.com > >>> > wrote: > >>> > >>> > Hi Patrick and Matei, > >>> > > >>> > Was trying out this and followed the quick start guide which > says do > >>> > sbt/sbt assembly, like few others I was also stuck for few > minutes on > >>> > linux. On the other hand if I use sbt/sbt assembly/assembly it is > >>> much > >>> > faster. > >>> > > >>> > Should we change the documentation to reflect this. It will not > be > >>> >>> great > >>> > for first time users to get stuck there. > >>> > > >>> > > >>> > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > >>> >>> matei.zaha...@gmail.com > >>> > wrote: > >>> > > >>> >> +1 > >>> >> > >>> >> Built and tested it on Mac OS X. > >>> >> > >>> >> Matei > >>> >> > >>> >> > >>> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell < > pwend...@gmail.com> > >>> wrote: > >>> >> > >>> >>> Please vote on releasing the following candidate as Apache > Spark > >>> >>> (incubating) version 0.8.1. > >>> >>> > >>> >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): > >>> >>> > >>> >> > >>> > >>> >>> > >>> > https
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
I also talked to a few people who got corrupted binaries when downloading from the people.apache HTTP. In that case the checksum failed but if they re-downloaded it worked. So maybe just re-download and try again? On Wed, Dec 11, 2013 at 3:15 PM, Patrick Wendell wrote: > Hey Tom, > > I re-verified the signatures and got someone else to do it. It seemed > fine. Here is what I did. > > gpg --recv-key 9E4FE3AF > wget > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc > wget > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz > gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz > gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID 9E4FE3AF > gpg: Good signature from "Patrick Wendell " > > On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra wrote: >> I don't know how to make sense of the numbers, but here's what I've got >> from a very small sample size. >> >> For both v0.8.0-incubating and v0.8.1-incubating, building separate >> assemblies is faster than `./sbt/sbt assembly` and the times for building >> separate assemblies for 0.8.0 and 0.8.1 are about the same. >> >> For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the >> sum of the separate assemblies. >> For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the >> sum of the separate assemblies. >> >> Weird. >> >> >> On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell wrote: >> >>> I'll +1 myself also. >>> >>> For anyone who has the slow build problem: does this issue happen when >>> building v0.8.0-incubating also? Trying to figure out whether it's >>> related to something we added in 0.8.1 or if it's a long standing >>> issue. >>> >>> - Patrick >>> >>> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia >>> wrote: >>> > Woah, weird, but definitely good to know. >>> > >>> > If you’re doing Spark development, there’s also a more convenient option >>> added by Shivaram in the master branch. You can do sbt assemble-deps to >>> package *just* the dependencies of each project in a special assembly JAR, >>> and then use sbt compile to update the code. This will use the classes >>> directly out of the target/scala-2.9.3/classes directories. You have to >>> redo assemble-deps only if your external dependencies change. >>> > >>> > Matei >>> > >>> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma >>> wrote: >>> > >>> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can >>> help. >>> >> Again this is not a blocker for the release from my side either. >>> >> >>> >> >>> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra >> >wrote: >>> >> >>> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` >>> takes >>> >>> a long, long, long time to complete (a MBP, in my case), building >>> three >>> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >>> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less >>> time. >>> >>> >>> >>> >>> >>> >>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < >>> scrapco...@gmail.com >>> wrote: >>> >>> >>> forgot to mention, after running sbt/sbt assembly/assembly running >>> >>> sbt/sbt >>> examples/assembly takes just 37s. Not to mention my hardware is not >>> >>> really >>> great. >>> >>> >>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < >>> scrapco...@gmail.com >>> > wrote: >>> >>> > Hi Patrick and Matei, >>> > >>> > Was trying out this and followed the quick start guide which says do >>> > sbt/sbt assembly, like few others I was also stuck for few minutes on >>> > linux. On the other hand if I use sbt/sbt assembly/assembly it is >>> much >>> > faster. >>> > >>> > Should we change the documentation to reflect this. It will not be >>> >>> great >>> > for first time users to get stuck there. >>> > >>> > >>> > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >>> >>> matei.zaha...@gmail.com >>> > wrote: >>> > >>> >> +1 >>> >> >>> >> Built and tested it on Mac OS X. >>> >> >>> >> Matei >>> >> >>> >> >>> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell >>> wrote: >>> >> >>> >>> Please vote on releasing the following candidate as Apache Spark >>> >>> (incubating) version 0.8.1. >>> >>> >>> >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): >>> >>> >>> >> >>> >>> >>> >>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >>> >>> >>> >>> The release files, including signatures, digests, etc can be found >>> >>> at: >>> >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >>> >>> >>> >>> Release artifacts are signed with the following key: >>> >>> https://people.apache.org/keys/committer/pwendell.asc >>> >>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hey Tom, I re-verified the signatures and got someone else to do it. It seemed fine. Here is what I did. gpg --recv-key 9E4FE3AF wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz.asc wget http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/spark-0.8.1-incubating.tgz gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz gpg: Signature made Tue 10 Dec 2013 02:53:15 PM PST using RSA key ID 9E4FE3AF gpg: Good signature from "Patrick Wendell " On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra wrote: > I don't know how to make sense of the numbers, but here's what I've got > from a very small sample size. > > For both v0.8.0-incubating and v0.8.1-incubating, building separate > assemblies is faster than `./sbt/sbt assembly` and the times for building > separate assemblies for 0.8.0 and 0.8.1 are about the same. > > For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the > sum of the separate assemblies. > For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the > sum of the separate assemblies. > > Weird. > > > On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell wrote: > >> I'll +1 myself also. >> >> For anyone who has the slow build problem: does this issue happen when >> building v0.8.0-incubating also? Trying to figure out whether it's >> related to something we added in 0.8.1 or if it's a long standing >> issue. >> >> - Patrick >> >> On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia >> wrote: >> > Woah, weird, but definitely good to know. >> > >> > If you’re doing Spark development, there’s also a more convenient option >> added by Shivaram in the master branch. You can do sbt assemble-deps to >> package *just* the dependencies of each project in a special assembly JAR, >> and then use sbt compile to update the code. This will use the classes >> directly out of the target/scala-2.9.3/classes directories. You have to >> redo assemble-deps only if your external dependencies change. >> > >> > Matei >> > >> > On Dec 11, 2013, at 1:04 AM, Prashant Sharma >> wrote: >> > >> >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can >> help. >> >> Again this is not a blocker for the release from my side either. >> >> >> >> >> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra > >wrote: >> >> >> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` >> takes >> >>> a long, long, long time to complete (a MBP, in my case), building >> three >> >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >> >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less >> time. >> >>> >> >>> >> >>> >> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < >> scrapco...@gmail.com >> wrote: >> >>> >> forgot to mention, after running sbt/sbt assembly/assembly running >> >>> sbt/sbt >> examples/assembly takes just 37s. Not to mention my hardware is not >> >>> really >> great. >> >> >> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < >> scrapco...@gmail.com >> > wrote: >> >> > Hi Patrick and Matei, >> > >> > Was trying out this and followed the quick start guide which says do >> > sbt/sbt assembly, like few others I was also stuck for few minutes on >> > linux. On the other hand if I use sbt/sbt assembly/assembly it is >> much >> > faster. >> > >> > Should we change the documentation to reflect this. It will not be >> >>> great >> > for first time users to get stuck there. >> > >> > >> > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >> >>> matei.zaha...@gmail.com >> > wrote: >> > >> >> +1 >> >> >> >> Built and tested it on Mac OS X. >> >> >> >> Matei >> >> >> >> >> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell >> wrote: >> >> >> >>> Please vote on releasing the following candidate as Apache Spark >> >>> (incubating) version 0.8.1. >> >>> >> >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): >> >>> >> >> >> >> >>> >> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >> >>> >> >>> The release files, including signatures, digests, etc can be found >> >>> at: >> >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >> >>> >> >>> Release artifacts are signed with the following key: >> >>> https://people.apache.org/keys/committer/pwendell.asc >> >>> >> >>> The staging repository for this release can be found at: >> >>> >> >> https://repository.apache.org/content/repositories/orgapachespark-040/ >> >>> >> >>> The documentation corresponding to this release can be found at: >> >>> >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ >> >>> >> >>> For information about the contents of this release see: >> >>>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
I don't know how to make sense of the numbers, but here's what I've got from a very small sample size. For both v0.8.0-incubating and v0.8.1-incubating, building separate assemblies is faster than `./sbt/sbt assembly` and the times for building separate assemblies for 0.8.0 and 0.8.1 are about the same. For v0.8.0-incubating, `./sbt/sbt assembly` takes about 2.5x as long as the sum of the separate assemblies. For v0.8.1-incubating, `./sbt/sbt assembly` takes almost 8x as long as the sum of the separate assemblies. Weird. On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell wrote: > I'll +1 myself also. > > For anyone who has the slow build problem: does this issue happen when > building v0.8.0-incubating also? Trying to figure out whether it's > related to something we added in 0.8.1 or if it's a long standing > issue. > > - Patrick > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia > wrote: > > Woah, weird, but definitely good to know. > > > > If you’re doing Spark development, there’s also a more convenient option > added by Shivaram in the master branch. You can do sbt assemble-deps to > package *just* the dependencies of each project in a special assembly JAR, > and then use sbt compile to update the code. This will use the classes > directly out of the target/scala-2.9.3/classes directories. You have to > redo assemble-deps only if your external dependencies change. > > > > Matei > > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > wrote: > > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can > help. > >> Again this is not a blocker for the release from my side either. > >> > >> > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra >wrote: > >> > >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` > takes > >>> a long, long, long time to complete (a MBP, in my case), building > three > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less > time. > >>> > >>> > >>> > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < > scrapco...@gmail.com > wrote: > >>> > forgot to mention, after running sbt/sbt assembly/assembly running > >>> sbt/sbt > examples/assembly takes just 37s. Not to mention my hardware is not > >>> really > great. > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < > scrapco...@gmail.com > > wrote: > > > Hi Patrick and Matei, > > > > Was trying out this and followed the quick start guide which says do > > sbt/sbt assembly, like few others I was also stuck for few minutes on > > linux. On the other hand if I use sbt/sbt assembly/assembly it is > much > > faster. > > > > Should we change the documentation to reflect this. It will not be > >>> great > > for first time users to get stuck there. > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > >>> matei.zaha...@gmail.com > > wrote: > > > >> +1 > >> > >> Built and tested it on Mac OS X. > >> > >> Matei > >> > >> > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell > wrote: > >> > >>> Please vote on releasing the following candidate as Apache Spark > >>> (incubating) version 0.8.1. > >>> > >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): > >>> > >> > > >>> > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > >>> > >>> The release files, including signatures, digests, etc can be found > >>> at: > >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > >>> > >>> Release artifacts are signed with the following key: > >>> https://people.apache.org/keys/committer/pwendell.asc > >>> > >>> The staging repository for this release can be found at: > >>> > > https://repository.apache.org/content/repositories/orgapachespark-040/ > >>> > >>> The documentation corresponding to this release can be found at: > >>> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > >>> > >>> For information about the contents of this release see: > >>> > >> > > >>> > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > >>> > >>> Please vote on releasing this package as Apache Spark > 0.8.1-incubating! > >>> > >>> The vote is open until Saturday, December 14th at 01:00 UTC and > >>> passes if a majority of at least 3 +1 PPMC votes are cast. > >>> > >>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating > >>> [ ] -1 Do not release this package because ... > >>> > >>> To learn more about Apache Spark, please see > >>> http://spark.incubator.apache.org/ > >> > >> > > > > > > -- > >>
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hey folks, I'm trying to verify the signature on the rc4 but am getting a BAD signature, is it working for others? Perhaps I messed up the import of the asc file. $ gpg --verify spark-0.8.1-incubating.tgz.asc spark-0.8.1-incubating.tgz gpg: Signature made Tue 10 Dec 2013 10:53:15 PM UTC using RSA key ID 9E4FE3AF gpg: BAD signature from "Patrick Wendell " Tom On Wednesday, December 11, 2013 1:50 PM, Patrick Wendell wrote: I'll +1 myself also. For anyone who has the slow build problem: does this issue happen when building v0.8.0-incubating also? Trying to figure out whether it's related to something we added in 0.8.1 or if it's a long standing issue. - Patrick On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia wrote: > Woah, weird, but definitely good to know. > > If you’re doing Spark development, there’s also a more convenient option > added by Shivaram in the master branch. You can do sbt assemble-deps to > package *just* the dependencies of each project in a special assembly JAR, > and then use sbt compile to update the code. This will use the classes > directly out of the target/scala-2.9.3/classes directories. You have to redo > assemble-deps only if your external dependencies change. > > Matei > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma wrote: > >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can help. >> Again this is not a blocker for the release from my side either. >> >> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra wrote: >> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes >>> a long, long, long time to complete (a MBP, in my case), building three >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time. >>> >>> >>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma >>> wrote: >>> forgot to mention, after running sbt/sbt assembly/assembly running >>> sbt/sbt examples/assembly takes just 37s. Not to mention my hardware is not >>> really great. On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma wrote: > Hi Patrick and Matei, > > Was trying out this and followed the quick start guide which says do > sbt/sbt assembly, like few others I was also stuck for few minutes on > linux. On the other hand if I use sbt/sbt assembly/assembly it is much > faster. > > Should we change the documentation to reflect this. It will not be >>> great > for first time users to get stuck there. > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >>> matei.zaha...@gmail.com > wrote: > >> +1 >> >> Built and tested it on Mac OS X. >> >> Matei >> >> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell wrote: >> >>> Please vote on releasing the following candidate as Apache Spark >>> (incubating) version 0.8.1. >>> >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): >>> >> >>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >>> >>> The release files, including signatures, digests, etc can be found >>> at: >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >>> >>> Release artifacts are signed with the following key: >>> https://people.apache.org/keys/committer/pwendell.asc >>> >>> The staging repository for this release can be found at: >>> https://repository.apache.org/content/repositories/orgapachespark-040/ >>> >>> The documentation corresponding to this release can be found at: >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ >>> >>> For information about the contents of this release see: >>> >> >>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 >>> >>> Please vote on releasing this package as Apache Spark 0.8.1-incubating! >>> >>> The vote is open until Saturday, December 14th at 01:00 UTC and >>> passes if a majority of at least 3 +1 PPMC votes are cast. >>> >>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating >>> [ ] -1 Do not release this package because ... >>> >>> To learn more about Apache Spark, please see >>> http://spark.incubator.apache.org/ >> >> > > > -- > s > -- s >>> >> >> >> >> -- >> s >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
On my Core 2 Duo (slower) Mac, I'm having the same problem with building v0.8.0-incubating as well. I'm suspecting the slow build problem is caused by sbt-assembly as reported here. https://github.com/sbt/sbt-assembly/issues/68 I tried to upgrade sbt-assembly to version 0.10.1 (and sbt to 0.13.0). But because of the "sbt-assembly 0.10.1 ==> sbt 0.13 ==> sbt-dependency-graph 0.7.4 ==> Scala 2.10" dependency, I wasn't able to use sbt-assembly 0.10.1 for this release. According to this thread ( https://github.com/sbt/sbt-assembly/issues/96 ), with sbt-assembly 0.9.2, setting "assemblyCacheOutput in assembly := false" might fix. I haven't verified, though. On Wed, Dec 11, 2013 at 11:49 AM, Patrick Wendell wrote: > I'll +1 myself also. > > For anyone who has the slow build problem: does this issue happen when > building v0.8.0-incubating also? Trying to figure out whether it's > related to something we added in 0.8.1 or if it's a long standing > issue. > > - Patrick > > On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia > wrote: > > Woah, weird, but definitely good to know. > > > > If you’re doing Spark development, there’s also a more convenient option > added by Shivaram in the master branch. You can do sbt assemble-deps to > package *just* the dependencies of each project in a special assembly JAR, > and then use sbt compile to update the code. This will use the classes > directly out of the target/scala-2.9.3/classes directories. You have to > redo assemble-deps only if your external dependencies change. > > > > Matei > > > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma > wrote: > > > >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can > help. > >> Again this is not a blocker for the release from my side either. > >> > >> > >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra >wrote: > >> > >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` > takes > >>> a long, long, long time to complete (a MBP, in my case), building > three > >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less > time. > >>> > >>> > >>> > >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma < > scrapco...@gmail.com > wrote: > >>> > forgot to mention, after running sbt/sbt assembly/assembly running > >>> sbt/sbt > examples/assembly takes just 37s. Not to mention my hardware is not > >>> really > great. > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma < > scrapco...@gmail.com > > wrote: > > > Hi Patrick and Matei, > > > > Was trying out this and followed the quick start guide which says do > > sbt/sbt assembly, like few others I was also stuck for few minutes on > > linux. On the other hand if I use sbt/sbt assembly/assembly it is > much > > faster. > > > > Should we change the documentation to reflect this. It will not be > >>> great > > for first time users to get stuck there. > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > >>> matei.zaha...@gmail.com > > wrote: > > > >> +1 > >> > >> Built and tested it on Mac OS X. > >> > >> Matei > >> > >> > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell > wrote: > >> > >>> Please vote on releasing the following candidate as Apache Spark > >>> (incubating) version 0.8.1. > >>> > >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): > >>> > >> > > >>> > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > >>> > >>> The release files, including signatures, digests, etc can be found > >>> at: > >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > >>> > >>> Release artifacts are signed with the following key: > >>> https://people.apache.org/keys/committer/pwendell.asc > >>> > >>> The staging repository for this release can be found at: > >>> > > https://repository.apache.org/content/repositories/orgapachespark-040/ > >>> > >>> The documentation corresponding to this release can be found at: > >>> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > >>> > >>> For information about the contents of this release see: > >>> > >> > > >>> > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > >>> > >>> Please vote on releasing this package as Apache Spark > 0.8.1-incubating! > >>> > >>> The vote is open until Saturday, December 14th at 01:00 UTC and > >>> passes if a majority of at least 3 +1 PPMC votes are cast. > >>> > >>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating > >>> [ ] -1 Do not release this package because ... > >>> > >>> To l
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
I'll +1 myself also. For anyone who has the slow build problem: does this issue happen when building v0.8.0-incubating also? Trying to figure out whether it's related to something we added in 0.8.1 or if it's a long standing issue. - Patrick On Wed, Dec 11, 2013 at 10:39 AM, Matei Zaharia wrote: > Woah, weird, but definitely good to know. > > If you’re doing Spark development, there’s also a more convenient option > added by Shivaram in the master branch. You can do sbt assemble-deps to > package *just* the dependencies of each project in a special assembly JAR, > and then use sbt compile to update the code. This will use the classes > directly out of the target/scala-2.9.3/classes directories. You have to redo > assemble-deps only if your external dependencies change. > > Matei > > On Dec 11, 2013, at 1:04 AM, Prashant Sharma wrote: > >> I hope this PR https://github.com/apache/incubator-spark/pull/252 can help. >> Again this is not a blocker for the release from my side either. >> >> >> On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra wrote: >> >>> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes >>> a long, long, long time to complete (a MBP, in my case), building three >>> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >>> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time. >>> >>> >>> >>> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma >>> wrote: >>> forgot to mention, after running sbt/sbt assembly/assembly running >>> sbt/sbt examples/assembly takes just 37s. Not to mention my hardware is not >>> really great. On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma wrote: > Hi Patrick and Matei, > > Was trying out this and followed the quick start guide which says do > sbt/sbt assembly, like few others I was also stuck for few minutes on > linux. On the other hand if I use sbt/sbt assembly/assembly it is much > faster. > > Should we change the documentation to reflect this. It will not be >>> great > for first time users to get stuck there. > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >>> matei.zaha...@gmail.com > wrote: > >> +1 >> >> Built and tested it on Mac OS X. >> >> Matei >> >> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell wrote: >> >>> Please vote on releasing the following candidate as Apache Spark >>> (incubating) version 0.8.1. >>> >>> The tag to be voted on is v0.8.1-incubating (commit b87d31d): >>> >> >>> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >>> >>> The release files, including signatures, digests, etc can be found >>> at: >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >>> >>> Release artifacts are signed with the following key: >>> https://people.apache.org/keys/committer/pwendell.asc >>> >>> The staging repository for this release can be found at: >>> https://repository.apache.org/content/repositories/orgapachespark-040/ >>> >>> The documentation corresponding to this release can be found at: >>> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ >>> >>> For information about the contents of this release see: >>> >> >>> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 >>> >>> Please vote on releasing this package as Apache Spark 0.8.1-incubating! >>> >>> The vote is open until Saturday, December 14th at 01:00 UTC and >>> passes if a majority of at least 3 +1 PPMC votes are cast. >>> >>> [ ] +1 Release this package as Apache Spark 0.8.1-incubating >>> [ ] -1 Do not release this package because ... >>> >>> To learn more about Apache Spark, please see >>> http://spark.incubator.apache.org/ >> >> > > > -- > s > -- s >>> >> >> >> >> -- >> s >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Woah, weird, but definitely good to know. If you’re doing Spark development, there’s also a more convenient option added by Shivaram in the master branch. You can do sbt assemble-deps to package *just* the dependencies of each project in a special assembly JAR, and then use sbt compile to update the code. This will use the classes directly out of the target/scala-2.9.3/classes directories. You have to redo assemble-deps only if your external dependencies change. Matei On Dec 11, 2013, at 1:04 AM, Prashant Sharma wrote: > I hope this PR https://github.com/apache/incubator-spark/pull/252 can help. > Again this is not a blocker for the release from my side either. > > > On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra wrote: > >> Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes >> a long, long, long time to complete (a MBP, in my case), building three >> separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt >> examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time. >> >> >> >> On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma >> wrote: >> >>> forgot to mention, after running sbt/sbt assembly/assembly running >> sbt/sbt >>> examples/assembly takes just 37s. Not to mention my hardware is not >> really >>> great. >>> >>> >>> On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma >>> wrote: >>> Hi Patrick and Matei, Was trying out this and followed the quick start guide which says do sbt/sbt assembly, like few others I was also stuck for few minutes on linux. On the other hand if I use sbt/sbt assembly/assembly it is much faster. Should we change the documentation to reflect this. It will not be >> great for first time users to get stuck there. On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < >> matei.zaha...@gmail.com wrote: > +1 > > Built and tested it on Mac OS X. > > Matei > > > On Dec 10, 2013, at 4:49 PM, Patrick Wendell >>> wrote: > >> Please vote on releasing the following candidate as Apache Spark >> (incubating) version 0.8.1. >> >> The tag to be voted on is v0.8.1-incubating (commit b87d31d): >> > >>> >> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >> >> The release files, including signatures, digests, etc can be found >> at: >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >> >> Release artifacts are signed with the following key: >> https://people.apache.org/keys/committer/pwendell.asc >> >> The staging repository for this release can be found at: >> >>> https://repository.apache.org/content/repositories/orgapachespark-040/ >> >> The documentation corresponding to this release can be found at: >> http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ >> >> For information about the contents of this release see: >> > >>> >> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 >> >> Please vote on releasing this package as Apache Spark >>> 0.8.1-incubating! >> >> The vote is open until Saturday, December 14th at 01:00 UTC and >> passes if a majority of at least 3 +1 PPMC votes are cast. >> >> [ ] +1 Release this package as Apache Spark 0.8.1-incubating >> [ ] -1 Do not release this package because ... >> >> To learn more about Apache Spark, please see >> http://spark.incubator.apache.org/ > > -- s >>> >>> >>> >>> -- >>> s >>> >> > > > > -- > s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
- Successfully built via sbt/sbt assembly/assembly on Mac OS X, as well as on a dev Ubuntu EC2 box - Successfully tested via sbt/sbt test locally - Successfully built and tested using mvn package locally - I've tested my own Spark jobs (built against 0.8.0-incubating) on this RC and all works fine, as well as tested with my job server (also built against 0.8.0-incubating) - Ran a few spark examples and the shell and PySpark shell - For my part, tested the MLlib implicit code I added, and checked docs I'm +1 On Wed, Dec 11, 2013 at 11:04 AM, Prashant Sharma wrote: > I hope this PR https://github.com/apache/incubator-spark/pull/252 can > help. > Again this is not a blocker for the release from my side either. > > > On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra >wrote: > > > Interesting, and confirmed: On my machine where `./sbt/sbt assembly` > takes > > a long, long, long time to complete (a MBP, in my case), building > three > > separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > > examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less > time. > > > > > > > > On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma > >wrote: > > > > > forgot to mention, after running sbt/sbt assembly/assembly running > > sbt/sbt > > > examples/assembly takes just 37s. Not to mention my hardware is not > > really > > > great. > > > > > > > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma > > >wrote: > > > > > > > Hi Patrick and Matei, > > > > > > > > Was trying out this and followed the quick start guide which says do > > > > sbt/sbt assembly, like few others I was also stuck for few minutes on > > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is > much > > > > faster. > > > > > > > > Should we change the documentation to reflect this. It will not be > > great > > > > for first time users to get stuck there. > > > > > > > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > > matei.zaha...@gmail.com > > > >wrote: > > > > > > > >> +1 > > > >> > > > >> Built and tested it on Mac OS X. > > > >> > > > >> Matei > > > >> > > > >> > > > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell > > > wrote: > > > >> > > > >> > Please vote on releasing the following candidate as Apache Spark > > > >> > (incubating) version 0.8.1. > > > >> > > > > >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d): > > > >> > > > > >> > > > > > > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > > >> > > > > >> > The release files, including signatures, digests, etc can be found > > at: > > > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > > > >> > > > > >> > Release artifacts are signed with the following key: > > > >> > https://people.apache.org/keys/committer/pwendell.asc > > > >> > > > > >> > The staging repository for this release can be found at: > > > >> > > > > https://repository.apache.org/content/repositories/orgapachespark-040/ > > > >> > > > > >> > The documentation corresponding to this release can be found at: > > > >> > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > > > >> > > > > >> > For information about the contents of this release see: > > > >> > > > > >> > > > > > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > > > >> > > > > >> > Please vote on releasing this package as Apache Spark > > > 0.8.1-incubating! > > > >> > > > > >> > The vote is open until Saturday, December 14th at 01:00 UTC and > > > >> > passes if a majority of at least 3 +1 PPMC votes are cast. > > > >> > > > > >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > > > >> > [ ] -1 Do not release this package because ... > > > >> > > > > >> > To learn more about Apache Spark, please see > > > >> > http://spark.incubator.apache.org/ > > > >> > > > >> > > > > > > > > > > > > -- > > > > s > > > > > > > > > > > > > > > > -- > > > s > > > > > > > > > -- > s >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
I hope this PR https://github.com/apache/incubator-spark/pull/252 can help. Again this is not a blocker for the release from my side either. On Wed, Dec 11, 2013 at 2:14 PM, Mark Hamstra wrote: > Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes > a long, long, long time to complete (a MBP, in my case), building three > separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt > examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time. > > > > On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma >wrote: > > > forgot to mention, after running sbt/sbt assembly/assembly running > sbt/sbt > > examples/assembly takes just 37s. Not to mention my hardware is not > really > > great. > > > > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma > >wrote: > > > > > Hi Patrick and Matei, > > > > > > Was trying out this and followed the quick start guide which says do > > > sbt/sbt assembly, like few others I was also stuck for few minutes on > > > linux. On the other hand if I use sbt/sbt assembly/assembly it is much > > > faster. > > > > > > Should we change the documentation to reflect this. It will not be > great > > > for first time users to get stuck there. > > > > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia < > matei.zaha...@gmail.com > > >wrote: > > > > > >> +1 > > >> > > >> Built and tested it on Mac OS X. > > >> > > >> Matei > > >> > > >> > > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell > > wrote: > > >> > > >> > Please vote on releasing the following candidate as Apache Spark > > >> > (incubating) version 0.8.1. > > >> > > > >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d): > > >> > > > >> > > > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > >> > > > >> > The release files, including signatures, digests, etc can be found > at: > > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > > >> > > > >> > Release artifacts are signed with the following key: > > >> > https://people.apache.org/keys/committer/pwendell.asc > > >> > > > >> > The staging repository for this release can be found at: > > >> > > > https://repository.apache.org/content/repositories/orgapachespark-040/ > > >> > > > >> > The documentation corresponding to this release can be found at: > > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > > >> > > > >> > For information about the contents of this release see: > > >> > > > >> > > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > > >> > > > >> > Please vote on releasing this package as Apache Spark > > 0.8.1-incubating! > > >> > > > >> > The vote is open until Saturday, December 14th at 01:00 UTC and > > >> > passes if a majority of at least 3 +1 PPMC votes are cast. > > >> > > > >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > > >> > [ ] -1 Do not release this package because ... > > >> > > > >> > To learn more about Apache Spark, please see > > >> > http://spark.incubator.apache.org/ > > >> > > >> > > > > > > > > > -- > > > s > > > > > > > > > > > -- > > s > > > -- s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Interesting, and confirmed: On my machine where `./sbt/sbt assembly` takes a long, long, long time to complete (a MBP, in my case), building three separate assemblies (`./sbt/sbt assembly/assembly`, `./sbt/sbt examples/assembly`, `./sbt/sbt tools/assembly`) takes much, much less time. On Wed, Dec 11, 2013 at 12:02 AM, Prashant Sharma wrote: > forgot to mention, after running sbt/sbt assembly/assembly running sbt/sbt > examples/assembly takes just 37s. Not to mention my hardware is not really > great. > > > On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma >wrote: > > > Hi Patrick and Matei, > > > > Was trying out this and followed the quick start guide which says do > > sbt/sbt assembly, like few others I was also stuck for few minutes on > > linux. On the other hand if I use sbt/sbt assembly/assembly it is much > > faster. > > > > Should we change the documentation to reflect this. It will not be great > > for first time users to get stuck there. > > > > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia >wrote: > > > >> +1 > >> > >> Built and tested it on Mac OS X. > >> > >> Matei > >> > >> > >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell > wrote: > >> > >> > Please vote on releasing the following candidate as Apache Spark > >> > (incubating) version 0.8.1. > >> > > >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d): > >> > > >> > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > >> > > >> > The release files, including signatures, digests, etc can be found at: > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > >> > > >> > Release artifacts are signed with the following key: > >> > https://people.apache.org/keys/committer/pwendell.asc > >> > > >> > The staging repository for this release can be found at: > >> > > https://repository.apache.org/content/repositories/orgapachespark-040/ > >> > > >> > The documentation corresponding to this release can be found at: > >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > >> > > >> > For information about the contents of this release see: > >> > > >> > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > >> > > >> > Please vote on releasing this package as Apache Spark > 0.8.1-incubating! > >> > > >> > The vote is open until Saturday, December 14th at 01:00 UTC and > >> > passes if a majority of at least 3 +1 PPMC votes are cast. > >> > > >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > >> > [ ] -1 Do not release this package because ... > >> > > >> > To learn more about Apache Spark, please see > >> > http://spark.incubator.apache.org/ > >> > >> > > > > > > -- > > s > > > > > > -- > s >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
That's a good point. Although it's definitely not a blocker for this release, it would be more user friendly to mention sbt/sbt assembly/assembly as well as Maven build instructions in the README and quick-start files (at least until the time-consuming sbt assembly process on a regular hardware gets resolved). I've been able to build RC4 (against Hadoop 2.2.0) with Maven on my slower Mac and Ubuntu 12.0.4 LTS successfully. On my faster Mac, sbt/sbt assembly has completed in 4min. Thanks for fixing the Maven build for Hadoop 2.2.0! On Tue, Dec 10, 2013 at 11:58 PM, Prashant Sharma wrote: > Hi Patrick and Matei, > > Was trying out this and followed the quick start guide which says do > sbt/sbt assembly, like few others I was also stuck for few minutes on > linux. On the other hand if I use sbt/sbt assembly/assembly it is much > faster. > > Should we change the documentation to reflect this. It will not be great > for first time users to get stuck there. > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia >wrote: > > > +1 > > > > Built and tested it on Mac OS X. > > > > Matei > > > > > > On Dec 10, 2013, at 4:49 PM, Patrick Wendell wrote: > > > > > Please vote on releasing the following candidate as Apache Spark > > > (incubating) version 0.8.1. > > > > > > The tag to be voted on is v0.8.1-incubating (commit b87d31d): > > > > > > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > > > > > The release files, including signatures, digests, etc can be found at: > > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > > > > > > Release artifacts are signed with the following key: > > > https://people.apache.org/keys/committer/pwendell.asc > > > > > > The staging repository for this release can be found at: > > > https://repository.apache.org/content/repositories/orgapachespark-040/ > > > > > > The documentation corresponding to this release can be found at: > > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > > > > > > For information about the contents of this release see: > > > > > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > > > > > > Please vote on releasing this package as Apache Spark 0.8.1-incubating! > > > > > > The vote is open until Saturday, December 14th at 01:00 UTC and > > > passes if a majority of at least 3 +1 PPMC votes are cast. > > > > > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > > > [ ] -1 Do not release this package because ... > > > > > > To learn more about Apache Spark, please see > > > http://spark.incubator.apache.org/ > > > > > > > -- > s >
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
forgot to mention, after running sbt/sbt assembly/assembly running sbt/sbt examples/assembly takes just 37s. Not to mention my hardware is not really great. On Wed, Dec 11, 2013 at 1:28 PM, Prashant Sharma wrote: > Hi Patrick and Matei, > > Was trying out this and followed the quick start guide which says do > sbt/sbt assembly, like few others I was also stuck for few minutes on > linux. On the other hand if I use sbt/sbt assembly/assembly it is much > faster. > > Should we change the documentation to reflect this. It will not be great > for first time users to get stuck there. > > > On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia wrote: > >> +1 >> >> Built and tested it on Mac OS X. >> >> Matei >> >> >> On Dec 10, 2013, at 4:49 PM, Patrick Wendell wrote: >> >> > Please vote on releasing the following candidate as Apache Spark >> > (incubating) version 0.8.1. >> > >> > The tag to be voted on is v0.8.1-incubating (commit b87d31d): >> > >> https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e >> > >> > The release files, including signatures, digests, etc can be found at: >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ >> > >> > Release artifacts are signed with the following key: >> > https://people.apache.org/keys/committer/pwendell.asc >> > >> > The staging repository for this release can be found at: >> > https://repository.apache.org/content/repositories/orgapachespark-040/ >> > >> > The documentation corresponding to this release can be found at: >> > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ >> > >> > For information about the contents of this release see: >> > >> https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 >> > >> > Please vote on releasing this package as Apache Spark 0.8.1-incubating! >> > >> > The vote is open until Saturday, December 14th at 01:00 UTC and >> > passes if a majority of at least 3 +1 PPMC votes are cast. >> > >> > [ ] +1 Release this package as Apache Spark 0.8.1-incubating >> > [ ] -1 Do not release this package because ... >> > >> > To learn more about Apache Spark, please see >> > http://spark.incubator.apache.org/ >> >> > > > -- > s > -- s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
Hi Patrick and Matei, Was trying out this and followed the quick start guide which says do sbt/sbt assembly, like few others I was also stuck for few minutes on linux. On the other hand if I use sbt/sbt assembly/assembly it is much faster. Should we change the documentation to reflect this. It will not be great for first time users to get stuck there. On Wed, Dec 11, 2013 at 9:54 AM, Matei Zaharia wrote: > +1 > > Built and tested it on Mac OS X. > > Matei > > > On Dec 10, 2013, at 4:49 PM, Patrick Wendell wrote: > > > Please vote on releasing the following candidate as Apache Spark > > (incubating) version 0.8.1. > > > > The tag to be voted on is v0.8.1-incubating (commit b87d31d): > > > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > > > The release files, including signatures, digests, etc can be found at: > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > > > > Release artifacts are signed with the following key: > > https://people.apache.org/keys/committer/pwendell.asc > > > > The staging repository for this release can be found at: > > https://repository.apache.org/content/repositories/orgapachespark-040/ > > > > The documentation corresponding to this release can be found at: > > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > > > > For information about the contents of this release see: > > > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > > > > Please vote on releasing this package as Apache Spark 0.8.1-incubating! > > > > The vote is open until Saturday, December 14th at 01:00 UTC and > > passes if a majority of at least 3 +1 PPMC votes are cast. > > > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > > [ ] -1 Do not release this package because ... > > > > To learn more about Apache Spark, please see > > http://spark.incubator.apache.org/ > > -- s
Re: [VOTE] Release Apache Spark 0.8.1-incubating (rc4)
+1 Built and tested it on Mac OS X. Matei On Dec 10, 2013, at 4:49 PM, Patrick Wendell wrote: > Please vote on releasing the following candidate as Apache Spark > (incubating) version 0.8.1. > > The tag to be voted on is v0.8.1-incubating (commit b87d31d): > https://git-wip-us.apache.org/repos/asf/incubator-spark/repo?p=incubator-spark.git;a=commit;h=b87d31dd8eb4b4e47c0138e9242d0dd6922c8c4e > > The release files, including signatures, digests, etc can be found at: > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4/ > > Release artifacts are signed with the following key: > https://people.apache.org/keys/committer/pwendell.asc > > The staging repository for this release can be found at: > https://repository.apache.org/content/repositories/orgapachespark-040/ > > The documentation corresponding to this release can be found at: > http://people.apache.org/~pwendell/spark-0.8.1-incubating-rc4-docs/ > > For information about the contents of this release see: > https://git-wip-us.apache.org/repos/asf?p=incubator-spark.git;a=blob;f=CHANGES.txt;h=ce0aeab524505b63c7999e0371157ac2def6fe1c;hb=branch-0.8 > > Please vote on releasing this package as Apache Spark 0.8.1-incubating! > > The vote is open until Saturday, December 14th at 01:00 UTC and > passes if a majority of at least 3 +1 PPMC votes are cast. > > [ ] +1 Release this package as Apache Spark 0.8.1-incubating > [ ] -1 Do not release this package because ... > > To learn more about Apache Spark, please see > http://spark.incubator.apache.org/
