Hi Iulian, Thanks for the hint. Just to make sure I understood: You actually used sbt assembly to build the spark for server binaries and not just the driver code compilation? Did you also had to change anything in particular in terms of merge conflicts or library exclusions for akka 2.4.0 compatibility?
Tnks. Rod -----Original Message----- From: "Iulian Dragoș" <[email protected]> Sent: 01/12/2015 17:28 To: "Akka User List" <[email protected]> Subject: Re: [akka-user] Building Spark with Akka 2.4.0 On Tuesday, December 1, 2015 at 6:09:39 PM UTC+1, drewhk wrote: On Tue, Dec 1, 2015 at 5:37 PM, Rodrigo Boavida <[email protected]> wrote: If I revert to Akka 2.3.11 on Scala 2.11 I do not have any problem running Spark.Seems something about the way Spark is calling the API that internally is creating a conflict? This happens while creating an actorSystem and some mailbox initialization if you look into the log I first posted. If there are any hints on where to look I will appreciate it. There is another possible explanation. I vaguely recall our Spark team having a similar issue and the cause was the maven-shade plugin. It seems like it modified the contents of the jar ("optimized" it or something) in a way that it break it. Indeed. The problem is with the maven shade plugin, it tries to do too many things when building the assembly, and breaks some classes. I managed to work around it by using `sbt assembly`, which is using another library to build the assembly. iulian I'll ask around, maybe someone else can help you here. -Endre Tnks, Rod On Tuesday, December 1, 2015 at 3:50:40 PM UTC, Rodrigo Boavida wrote: Hi Endre, Spark is supposed to be binary compatible with Scala 2.11. It's actually documented. Here is where they document it, and I'm following the instructions http://spark.apache.org/docs/latest/building-spark.html I actually found that I was missing the -Dscala-2.11 in the maven command when running the dependency, that's why I was getting the previous error. In attach is the result of running the dependencies. tnks, Rod On Tuesday, December 1, 2015 at 3:36:51 PM UTC, drewhk wrote: 2.4.0 is not released against 2.10, so that explains it. You probably cannot update to Akka 2.4.0 until Spark is updated to Scala 2.11. -Endre On Tue, Dec 1, 2015 at 4:32 PM, Rodrigo Boavida <[email protected]> wrote: Here is complete dependency resolution. Seems 2.10 dependencies are being pulled from spark-core. Downloading: https://repo1.maven.org/maven2/com/twitter/chill_2.10/0.5.0/chill_2.10-0.5.0.pom Downloaded: https://repo1.maven.org/maven2/com/twitter/chill_2.10/0.5.0/chill_2.10-0.5.0.pom (3 KB at 30.5 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.2.0/hadoop-client-2.2.0.pom Downloaded: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.2.0/hadoop-client-2.2.0.pom (11 KB at 117.4 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-project-dist/2.2.0/hadoop-project-dist-2.2.0.pom Downloaded: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-project-dist/2.2.0/hadoop-project-dist-2.2.0.pom (18 KB at 250.3 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-project/2.2.0/hadoop-project-2.2.0.pom Downloaded: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-project/2.2.0/hadoo-project-2.2.0.pom (37 KB at 421.9 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-main/2.2.0/hadoop-main-2.2.0.pom Downloaded: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-main/2.2.0/hadoop-main-2.2.0.pom (17 KB at 224.7 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/2.2.0/hadoop-common-2.2.0.pom Downloaded: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/2.2.0/hadoop-common-2.2.0.pom (27 KB at 312.4 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-annotations/2.2.0/hadoop-annotations-2.2.0.pom Downloaded: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-annotations/2.2.0/hadoop-annotations-2.2.0.pom (3 KB at 39.5 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/commons/commons-math/2.1/commons-math-2.1.pom Downloaded: https://repo1.maven.org/maven2/org/apache/commons/commons-math/2.1/commons-math-2.1.pom (11 KB at 140.0 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/commons/commons-parent/14/commons-parent-14.pom Downloaded: https://repo1.maven.org/maven2/org/apache/commons/commons-parent/14/commons-parent-14.pom (31 KB at 369.3 KB/sec) Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/2.2.0/hadoop-auth-2.2.0.pom Downloaded: <a href="https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-auth/2.2.0/hadoop-auth-2.2.0.pom"; rel="nofollow" target="_ [The entire original message is not included.] -- >>>>>>>>>> Read the docs: http://akka.io/docs/ >>>>>>>>>> Check the FAQ: >>>>>>>>>> http://doc.akka.io/docs/akka/current/additional/faq.html >>>>>>>>>> Search the archives: https://groups.google.com/group/akka-user --- You received this message because you are subscribed to the Google Groups "Akka User List" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/akka-user. For more options, visit https://groups.google.com/d/optout.
