Thank u Patrick. -----Original Message----- From: Patrick Wendell [mailto:[email protected]] Sent: Friday, January 17, 2014 11:54 PM To: [email protected] Subject: Re: SparkException: Expect only DirectTaskResults when using localScheduler()
This is a bug that was fixed and will be part of 0.8.2: https://github.com/apache/incubator-spark/pull/281 A workaround is setting the akka frame size to be larger using spark.akka.frameSize. The issue is that we added a mechanism to circumvent akka for large task results, but the implementation assumed this would never happen in local mode. That assumption turned out to be wrong, so it was patched in #281. - Patrick On Fri, Jan 17, 2014 at 9:53 PM, Nick Pentreath <[email protected]> wrote: > I'm also running into this issue. If it is related to a setting then > the error message should surely be cleaned up a bit to be more helpful > in suggesting a fix? > - > Sent from Mailbox for iPhone > > > On Sat, Jan 18, 2014 at 2:35 AM, [email protected] > <[email protected]> wrote: >> >> Hi, >> >> >> >> Can someone please explain what it means the following exception >> 'SparkException: Expect only DirectTaskResults when using localScheduler()'? >> >> This is related to spark.akka.frameSize? >> >> I am running single node local spark 0.8.1 >> >> >> >> Thanks, >> >> Hussam > >
