Updates: 1. Fix for the ASM problem<https://spark-project.atlassian.net/browse/SPARK-782>that Kevin mentioned is already in Spark 0.9.1 RC2 2. Fix for pyspark's RDD.top() <https://spark-project.atlassian.net/browse/SPARK-1322>that Patrick mentioned has been pulled into branch 0.9. This will get into the next RC if there is one.
TD On Wed, Mar 26, 2014 at 9:21 AM, Patrick Wendell <pwend...@gmail.com> wrote: > Hey TD, > > This one we just merged into master this morning: > https://spark-project.atlassian.net/browse/SPARK-1322 > > It should definitely go into the 0.9 branch because there was a bug in the > semantics of top() which at this point is unreleased in Python. > > I didn't backport it yet because I figured you might want to do this at a > specific time. So please go ahead and backport it. Not sure whether this > warrants another RC. > > - Patrick > > > On Tue, Mar 25, 2014 at 10:47 PM, Mridul Muralidharan <mri...@gmail.com > >wrote: > > > On Wed, Mar 26, 2014 at 10:53 AM, Tathagata Das > > <tathagata.das1...@gmail.com> wrote: > > > PR 159 seems like a fairly big patch to me. And quite recent, so its > > impact > > > on the scheduling is not clear. It may also depend on other changes > that > > > may have gotten into the DAGScheduler but not pulled into branch 0.9. I > > am > > > not sure it is a good idea to pull that in. We can pull those changes > > later > > > for 0.9.2 if required. > > > > > > There is no impact on scheduling : it only has an impact on error > > handling - it ensures that you can actually use spark on yarn in > > multi-tennent clusters more reliably. > > Currently, any reasonably long running job (30 mins+) working on non > > trivial dataset will fail due to accumulated failures in spark. > > > > > > Regards, > > Mridul > > > > > > > > > > TD > > > > > > > > > > > > > > > On Tue, Mar 25, 2014 at 8:44 PM, Mridul Muralidharan <mri...@gmail.com > > >wrote: > > > > > >> Forgot to mention this in the earlier request for PR's. > > >> If there is another RC being cut, please add > > >> https://github.com/apache/spark/pull/159 to it too (if not done > > >> already !). > > >> > > >> Thanks, > > >> Mridul > > >> > > >> On Thu, Mar 20, 2014 at 5:37 AM, Tathagata Das > > >> <tathagata.das1...@gmail.com> wrote: > > >> > Hello everyone, > > >> > > > >> > Since the release of Spark 0.9, we have received a number of > important > > >> bug > > >> > fixes and we would like to make a bug-fix release of Spark 0.9.1. We > > are > > >> > going to cut a release candidate soon and we would love it if people > > test > > >> > it out. We have backported several bug fixes into the 0.9 and > updated > > >> JIRA > > >> > accordingly< > > >> > > > https://spark-project.atlassian.net/browse/SPARK-1275?jql=project%20in%20(SPARK%2C%20BLINKDB%2C%20MLI%2C%20MLLIB%2C%20SHARK%2C%20STREAMING%2C%20GRAPH%2C%20TACHYON)%20AND%20fixVersion%20%3D%200.9.1%20AND%20status%20in%20(Resolved%2C%20Closed) > > >> >. > > >> > Please let me know if there are fixes that were not backported but > you > > >> > would like to see them in 0.9.1. > > >> > > > >> > Thanks! > > >> > > > >> > TD > > >> > > >
