Hi Developers,
In the next few days we are planning to merge Scala 2.10 support into
Spark. For those that haven't been following this, Prashant Sharma has been
maintaining the scala-2.10 branch of Spark for several months. This branch
is current with master and has been reviewed for merging:
htt
+1, built and tested on linux.
On Thu, Dec 12, 2013 at 5:57 AM, Patrick Wendell wrote:
> I also talked to a few people who got corrupted binaries when
> downloading from the people.apache HTTP. In that case the checksum
> failed but if they re-downloaded it worked. So maybe just re-download
> a
Done.
--
Olivier
Hi Patrick
What does that means for drop YARN 2.2? seems codes are still there.
You mean if build upon 2.2 it will break, and won't and work right? Since the
home made akka build on scala 2.10 are not there. While, if for this case, can
we just use akka 2.3-M1 which run on protobuf 2.5
Hey Raymond,
This won't work because AFAIK akka 2.3-M1 is not binary compatible with
akka 2.2.3 (right?). For all of the non-yarn 2.2 versions we need to still
use the older protobuf library, so we'd need to support both.
I'd also be concerned about having a reference to a non-released version of
Also - the code is still there because of a recent merge that took in some
newer changes... we'll be removing it for the final merge.
On Thu, Dec 12, 2013 at 1:12 AM, Patrick Wendell wrote:
> Hey Raymond,
>
> This won't work because AFAIK akka 2.3-M1 is not binary compatible with
> akka 2.2.3 (
Hey Raymond,
I just gave what you said a try but have no intentions of maintaining it.
You might want it helpful
https://github.com/ScrapCodes/incubator-spark/tree/yarn-2.2 incase you are
willing to maintain it.
For the record, above branch is updated to master with scala 2.10 and uses
akka 2.3-M
I re-downloaded the source tarball and it works now.
Tom
On Wednesday, December 11, 2013 6:27 PM, Patrick Wendell
wrote:
I also talked to a few people who got corrupted binaries when
downloading from the people.apache HTTP. In that case the checksum
failed but if they re-downloaded it worke
+1.
Built spark on yarn for both hadoop 0.23 and hadoop 2.2.0 on redhat linux using
maven. Ran some tests on both a secure Hadoop 0.23 cluster and a secure Hadoop
2.2.0 cluster. Verified signatures and md5.
Tom
On Tuesday, December 10, 2013 6:49 PM, Patrick Wendell
wrote:
Please vote
Mark,
Thanks. The FutureAction API looks awesome.
On Mon, Dec 9, 2013 at 9:31 AM, Mark Hamstra wrote:
> Spark has already supported async jobs for awhile now --
> https://github.com/apache/incubator-spark/pull/29, and they even work
> correctly after https://github.com/apache/incubator-spark/p
I'd be personally fine with a standard workflow of assemble-deps +
packaging just the Spark files as separate packages, if it speeds up
everyone's development time.
On Wed, Dec 11, 2013 at 1:10 PM, Mark Hamstra wrote:
> I don't know how to make sense of the numbers, but here's what I've got
> fr
Nick, have you tried using the latest Scala plug-in, which features native
SBT project imports? ie you no longer need to run gen-idea.
On Sat, Dec 7, 2013 at 4:15 AM, Nick Pentreath wrote:
> Hi Spark Devs,
>
> Hoping someone cane help me out. No matter what I do, I cannot get Intellij
> to bui
I'm having fun with it. And it's almost all Reynold's work, so I can't
take credit for it.
On Thu, Dec 12, 2013 at 8:56 AM, Evan Chan wrote:
> Mark,
>
> Thanks. The FutureAction API looks awesome.
>
>
> On Mon, Dec 9, 2013 at 9:31 AM, Mark Hamstra >wrote:
>
> > Spark has already supported as
Hi Patrick
So what's the plan for support Yarn 2.2 in 0.9? As far as I can see, if
you want to support both 2.2 and 2.0 , due to protobuf version incompatible
issue. You need two version of akka anyway.
Akka 2.3-M1 looks like have a little bit change in API, we probably
could i
Hey Reymond,
Let's move this discussion out of this thread and into the associated JIRA.
I'll write up our current approach over there.
https://spark-project.atlassian.net/browse/SPARK-995
- Patrick
On Thu, Dec 12, 2013 at 5:56 PM, Liu, Raymond wrote:
> Hi Patrick
>
> So what's the p
15 matches
Mail list logo
