I've run spark 0.9.1 on 0.18.0. I did need to patch Spark in order for it to work.
diff --git a/pom.xml b/pom.xml index e82d9e8..b1bc684 100644 --- a/pom.xml +++ b/pom.xml @@ -109,13 +109,13 @@ <scala.version>2.10.3</scala.version> <scala.binary.version>2.10</scala.binary.version> - <mesos.version>0.13.0</mesos.version> + <mesos.version>0.18.0-rc6</mesos.version> <akka.group>org.spark-project.akka</akka.group> <akka.version>2.2.3-shaded-protobuf</akka.version> <slf4j.version>1.7.2</slf4j.version> <log4j.version>1.2.17</log4j.version> <hadoop.version>1.0.4</hadoop.version> - <protobuf.version>2.4.1</protobuf.version> + <protobuf.version>2.5.0</protobuf.version> <yarn.version>0.23.7</yarn.version> <hbase.version>0.94.6</hbase.version> This has been my standard pattern, but I also found wonkiness in their scheduler that I patched. Cheers, Tim ----- Original Message ----- > From: "Steven Cox" <[email protected]> > To: [email protected] > Sent: Tuesday, April 22, 2014 7:05:53 PM > Subject: Spark 0.9.1 core dump on Mesos > So I tried a fix found on the list... changing the mesos version from 0.13.0 > to 0.18.0. This apparently worked for someone else on the list in their > configuration. > In mine, it still core dumps like this: > https://gist.github.com/stevencox/11002498 > In this environment: > Ubuntu 13.10 > Mesos 0.18.0 > Spark 0.9.1 > JDK 1.7.0_45 > Scala 2.10.1 > I've tried multiple JDK implementations/versions. All core dump. > I've switched back to Mesos 0.13.0 per the Spark on mesos docs. Still core > dumps. > What am I missing? -- Cheers, Tim Freedom, Features, Friends, First -> Fedora https://fedoraproject.org/wiki/SIGs/bigdata
