Hi Patrick, Although we are able to use Spark 1.1.0 with Play 2.2.x, as you mentioned, Akka incompatibility prevents us from using Spark with the current stable releases of Play (2.3.6) and Akka (2.3.7). Are there any plans to address this issue in Spark 1.2?
Thanks, Mohammed From: John Meehan [mailto:[email protected]] Sent: Tuesday, November 11, 2014 11:35 PM To: Mohammed Guller Cc: Patrick Wendell; Akshat Aranya; [email protected] Subject: Re: Spark and Play You can also build a Play 2.2.x + Spark 1.1.0 fat jar with sbt-assembly for, e.g. yarn-client support or using with spark-shell for debugging: play.Project.playScalaSettings libraryDependencies ~= { _ map { case m if m.organization == "com.typesafe.play" => m.exclude("commons-logging", "commons-logging") case m => m }} assemblySettings test in assembly := {} mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) => { case m if m.toLowerCase.endsWith("manifest.mf") => MergeStrategy.discard case m if m.startsWith("META-INF") => MergeStrategy.discard case PathList("javax", "servlet", xs @ _*) => MergeStrategy.first case PathList("org", "apache", xs @ _*) => MergeStrategy.first case PathList("org", "jboss", xs @ _*) => MergeStrategy.first case PathList("org", "slf4j", xs @ _*) => MergeStrategy.discard case "about.html" => MergeStrategy.rename case "reference.conf" => MergeStrategy.concat case _ => MergeStrategy.first } } On Tue, Nov 11, 2014 at 3:04 PM, Mohammed Guller <[email protected]<mailto:[email protected]>> wrote: Actually, it is possible to integrate Spark 1.1.0 with Play 2.2.x Here is a sample build.sbt file: name := """xyz""" version := "0.1 " scalaVersion := "2.10.4" libraryDependencies ++= Seq( jdbc, anorm, cache, "org.apache.spark" %% "spark-core" % "1.1.0", "com.typesafe.akka" %% "akka-actor" % "2.2.3", "com.typesafe.akka" %% "akka-slf4j" % "2.2.3", "org.apache.spark" %% "spark-sql" % "1.1.0" ) play.Project.playScalaSettings Mohammed -----Original Message----- From: Patrick Wendell [mailto:[email protected]<mailto:[email protected]>] Sent: Tuesday, November 11, 2014 2:06 PM To: Akshat Aranya Cc: [email protected]<mailto:[email protected]> Subject: Re: Spark and Play Hi There, Because Akka versions are not binary compatible with one another, it might not be possible to integrate Play with Spark 1.1.0. - Patrick On Tue, Nov 11, 2014 at 8:21 AM, Akshat Aranya <[email protected]<mailto:[email protected]>> wrote: > Hi, > > Sorry if this has been asked before; I didn't find a satisfactory > answer when searching. How can I integrate a Play application with > Spark? I'm getting into issues of akka-actor versions. Play 2.2.x > uses akka-actor 2.0, whereas Play 2.3.x uses akka-actor 2.3.4, neither > of which work fine with Spark 1.1.0. Is there something I should do > with libraryDependencies in my build.sbt to make it work? > > Thanks, > Akshat --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]> --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected]<mailto:[email protected]> For additional commands, e-mail: [email protected]<mailto:[email protected]>
