I assume we will have an rc10 to fix the issues Matei found? Tom
On Sunday, May 18, 2014 9:08 PM, Patrick Wendell <pwend...@gmail.com> wrote: Hey Matei - the issue you found is not related to security. This patch a few days ago broke builds for Hadoop 1 with YARN support enabled. The patch directly altered the way we deal with commons-lang dependency, which is what is at the base of this stack trace. https://github.com/apache/spark/pull/754 - Patrick On Sun, May 18, 2014 at 5:28 PM, Matei Zaharia <matei.zaha...@gmail.com> wrote: > Alright, I've opened https://github.com/apache/spark/pull/819 with the > Windows fixes. I also found one other likely bug, > https://issues.apache.org/jira/browse/SPARK-1875, in the binary packages for > Hadoop1 built in this RC. I think this is due to Hadoop 1's security code > depending on a different version of org.apache.commons than Hadoop 2, but it > needs investigation. Tom, any thoughts on this? > > Matei > > On May 18, 2014, at 12:33 PM, Matei Zaharia <matei.zaha...@gmail.com> wrote: > >> I took the always fun task of testing it on Windows, and unfortunately, I >> found some small problems with the prebuilt packages due to recent changes >> to the launch scripts: bin/spark-class2.cmd looks in ./jars instead of ./lib >> for the assembly JAR, and bin/run-example2.cmd doesn't quite match the >> master-setting behavior of the Unix based one. I'll send a pull request to >> fix them soon. >> >> Matei >> >> >> On May 17, 2014, at 11:32 AM, Sandy Ryza <sandy.r...@cloudera.com> wrote: >> >>> +1 >>> >>> Reran my tests from rc5: >>> >>> * Built the release from source. >>> * Compiled Java and Scala apps that interact with HDFS against it. >>> * Ran them in local mode. >>> * Ran them against a pseudo-distributed YARN cluster in both yarn-client >>> mode and yarn-cluster mode. >>> >>> >>> On Sat, May 17, 2014 at 10:08 AM, Andrew Or <and...@databricks.com> wrote: >>> >>>> +1 >>>> >>>> >>>> 2014-05-17 8:53 GMT-07:00 Mark Hamstra <m...@clearstorydata.com>: >>>> >>>>> +1 >>>>> >>>>> >>>>> On Sat, May 17, 2014 at 12:58 AM, Patrick Wendell <pwend...@gmail.com >>>>>> wrote: >>>>> >>>>>> I'll start the voting with a +1. >>>>>> >>>>>> On Sat, May 17, 2014 at 12:58 AM, Patrick Wendell <pwend...@gmail.com> >>>>>> wrote: >>>>>>> Please vote on releasing the following candidate as Apache Spark >>>>> version >>>>>> 1.0.0! >>>>>>> This has one bug fix and one minor feature on top of rc8: >>>>>>> SPARK-1864: https://github.com/apache/spark/pull/808 >>>>>>> SPARK-1808: https://github.com/apache/spark/pull/799 >>>>>>> >>>>>>> The tag to be voted on is v1.0.0-rc9 (commit 920f947): >>>>>>> >>>>>> >>>>> >>>> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=920f947eb5a22a679c0c3186cf69ee75f6041c75 >>>>>>> >>>>>>> The release files, including signatures, digests, etc. can be found >>>> at: >>>>>>> http://people.apache.org/~pwendell/spark-1.0.0-rc9/ >>>>>>> >>>>>>> Release artifacts are signed with the following key: >>>>>>> https://people.apache.org/keys/committer/pwendell.asc >>>>>>> >>>>>>> The staging repository for this release can be found at: >>>>>>> >>>>> https://repository.apache.org/content/repositories/orgapachespark-1017/ >>>>>>> >>>>>>> The documentation corresponding to this release can be found at: >>>>>>> http://people.apache.org/~pwendell/spark-1.0.0-rc9-docs/ >>>>>>> >>>>>>> Please vote on releasing this package as Apache Spark 1.0.0! >>>>>>> >>>>>>> The vote is open until Tuesday, May 20, at 08:56 UTC and passes if >>>>>>> amajority of at least 3 +1 PMC votes are cast. >>>>>>> >>>>>>> [ ] +1 Release this package as Apache Spark 1.0.0 >>>>>>> [ ] -1 Do not release this package because ... >>>>>>> >>>>>>> To learn more about Apache Spark, please see >>>>>>> http://spark.apache.org/ >>>>>>> >>>>>>> == API Changes == >>>>>>> We welcome users to compile Spark applications against 1.0. There are >>>>>>> a few API changes in this release. Here are links to the associated >>>>>>> upgrade guides - user facing changes have been kept as small as >>>>>>> possible. >>>>>>> >>>>>>> changes to ML vector specification: >>>>>>> >>>>>> >>>>> >>>> http://people.apache.org/~pwendell/spark-1.0.0-rc8-docs/mllib-guide.html#from-09-to-10 >>>>>>> >>>>>>> changes to the Java API: >>>>>>> >>>>>> >>>>> >>>> http://people.apache.org/~pwendell/spark-1.0.0-rc8-docs/java-programming-guide.html#upgrading-from-pre-10-versions-of-spark >>>>>>> >>>>>>> changes to the streaming API: >>>>>>> >>>>>> >>>>> >>>> http://people.apache.org/~pwendell/spark-1.0.0-rc8-docs/streaming-programming-guide.html#migration-guide-from-091-or-below-to-1x >>>>>>> >>>>>>> changes to the GraphX API: >>>>>>> >>>>>> >>>>> >>>> http://people.apache.org/~pwendell/spark-1.0.0-rc8-docs/graphx-programming-guide.html#upgrade-guide-from-spark-091 >>>>>>> >>>>>>> coGroup and related functions now return Iterable[T] instead of >>>> Seq[T] >>>>>>> ==> Call toSeq on the result to restore the old behavior >>>>>>> >>>>>>> SparkContext.jarOfClass returns Option[String] instead of Seq[String] >>>>>>> ==> Call toSeq on the result to restore old behavior >>>>>> >>>>> >>>> >> >
