Hey Gary, Why do you think the akka frame size changed? It didn't change - we added some fixes for cases where users were setting non-default values.
On Fri, Jul 11, 2014 at 9:31 AM, Gary Malouf <malouf.g...@gmail.com> wrote: > Hi Matei, > > We have not had time to re-deploy the rc today, but one thing that jumps > out is the shrinking of the default akka frame size from 10MB to around > 128KB by default. That is my first suspicion for our issue - could imagine > that biting others as well. > > I'll try to re-test that today - either way, understand moving forward at > this point. > > Gary > > > On Fri, Jul 11, 2014 at 12:08 PM, Matei Zaharia <matei.zaha...@gmail.com> > wrote: > >> Unless you can diagnose the problem quickly, Gary, I think we need to go >> ahead with this release as is. This release didn't touch the Mesos support >> as far as I know, so the problem might be a nondeterministic issue with >> your application. But on the other hand the release does fix some critical >> bugs that affect all users. We can always do 1.0.2 later if we discover a >> problem. >> >> Matei >> >> On Jul 10, 2014, at 9:40 PM, Patrick Wendell <pwend...@gmail.com> wrote: >> >> > Hey Gary, >> > >> > The vote technically doesn't close until I send the vote summary >> > e-mail, but I was planning to close and package this tonight. It's too >> > bad if there is a regression, it might be worth holding the release >> > but it really requires narrowing down the issue to get more >> > information about the scope and severity. Could you fork another >> > thread for this? >> > >> > - Patrick >> > >> > On Thu, Jul 10, 2014 at 6:28 PM, Gary Malouf <malouf.g...@gmail.com> >> wrote: >> >> -1 I honestly do not know the voting rules for the Spark community, so >> >> please excuse me if I am out of line or if Mesos compatibility is not a >> >> concern at this point. >> >> >> >> We just tried to run this version built against 2.3.0-cdh5.0.2 on mesos >> >> 0.18.2. All of our jobs with data above a few gigabytes hung >> indefinitely. >> >> Downgrading back to the 1.0.0 stable release of Spark built the same way >> >> worked for us. >> >> >> >> >> >> On Mon, Jul 7, 2014 at 5:17 PM, Tom Graves <tgraves...@yahoo.com.invalid >> > >> >> wrote: >> >> >> >>> +1. Ran some Spark on yarn jobs on a hadoop 2.4 cluster with >> >>> authentication on. >> >>> >> >>> Tom >> >>> >> >>> >> >>> On Friday, July 4, 2014 2:39 PM, Patrick Wendell <pwend...@gmail.com> >> >>> wrote: >> >>> >> >>> >> >>> >> >>> Please vote on releasing the following candidate as Apache Spark >> version >> >>> 1.0.1! >> >>> >> >>> The tag to be voted on is v1.0.1-rc1 (commit 7d1043c): >> >>> >> >>> >> https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=7d1043c99303b87aef8ee19873629c2bfba4cc78 >> >>> >> >>> The release files, including signatures, digests, etc. can be found at: >> >>> http://people.apache.org/~pwendell/spark-1.0.1-rc2/ >> >>> >> >>> Release artifacts are signed with the following key: >> >>> https://people.apache.org/keys/committer/pwendell.asc >> >>> >> >>> The staging repository for this release can be found at: >> >>> >> https://repository.apache.org/content/repositories/orgapachespark-1021/ >> >>> >> >>> The documentation corresponding to this release can be found at: >> >>> http://people.apache.org/~pwendell/spark-1.0.1-rc2-docs/ >> >>> >> >>> Please vote on releasing this package as Apache Spark 1.0.1! >> >>> >> >>> The vote is open until Monday, July 07, at 20:45 UTC and passes if >> >>> a majority of at least 3 +1 PMC votes are cast. >> >>> >> >>> [ ] +1 Release this package as Apache Spark 1.0.1 >> >>> [ ] -1 Do not release this package because ... >> >>> >> >>> To learn more about Apache Spark, please see >> >>> http://spark.apache.org/ >> >>> >> >>> === Differences from RC1 === >> >>> This release includes only one "blocking" patch from rc1: >> >>> https://github.com/apache/spark/pull/1255 >> >>> >> >>> There are also smaller fixes which came in over the last week. >> >>> >> >>> === About this release === >> >>> This release fixes a few high-priority bugs in 1.0 and has a variety >> >>> of smaller fixes. The full list is here: http://s.apache.org/b45. Some >> >>> of the more visible patches are: >> >>> >> >>> SPARK-2043: ExternalAppendOnlyMap doesn't always find matching keys >> >>> SPARK-2156 and SPARK-1112: Issues with jobs hanging due to akka frame >> size. >> >>> SPARK-1790: Support r3 instance types on EC2. >> >>> >> >>> This is the first maintenance release on the 1.0 line. We plan to make >> >>> additional maintenance releases as new fixes come in. >> >>> >> >>
