On Tue, Dec 30, 2014 at 11:45 AM, xhudik <xhu...@gmail.com> wrote: > I also tried Java8 and scala 2.11 (no -Dscala.usejavacp=true), but I failed > for some other problem: > > /mvn -Pyarn -Phadoop-2.5 -Dhadoop.version=2.5.0 -Dscala-2.11 -X -DskipTests > clean package
There is no "hadoop-2.5" profile, as the output tells you. Use "-Phadoop-2.4" for 2.4+. That shouldn't be the issue though. I'd set "-Pscala-2.11" to activate Scala 2.11 but it should also be activated by the property. I'm also curious about this problem, but I think the answer is roughly this: you can't build the Scala 2.11 build from source unless you also hand-modify the artifacts to have "_2.11" in the name. This is due to some Maven complexities. There is a script called change-version-to-2.11.sh that helps to do this. Without that, you're not using locally-built artifacts at all. And your assembly can't find things like the REPL which are not otherwise published to Maven. > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Spark Project Parent POM ........................... SUCCESS [ 14.453 > s] > [INFO] Spark Project Core ................................. SUCCESS [ 47.508 > s] > [INFO] Spark Project Bagel ................................ SUCCESS [ 3.646 > s] > [INFO] Spark Project GraphX ............................... SUCCESS [ 5.533 > s] > [INFO] Spark Project ML Library ........................... SUCCESS [ 12.715 > s] > [INFO] Spark Project Tools ................................ SUCCESS [ 1.854 > s] > [INFO] Spark Project Networking ........................... SUCCESS [ 6.580 > s] > [INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 5.290 > s] > [INFO] Spark Project Streaming ............................ SUCCESS [ 10.846 > s] > [INFO] Spark Project Catalyst ............................. SUCCESS [ 8.296 > s] > [INFO] Spark Project SQL .................................. SUCCESS [ 12.921 > s] > [INFO] Spark Project Hive ................................. SUCCESS [ 28.931 > s] > [INFO] Spark Project Assembly ............................. FAILURE [01:09 > min] > [INFO] Spark Project External Twitter ..................... SKIPPED > [INFO] Spark Project External Flume ....................... SKIPPED > [INFO] Spark Project External Flume Sink .................. SKIPPED > [INFO] Spark Project External MQTT ........................ SKIPPED > [INFO] Spark Project External ZeroMQ ...................... SKIPPED > [INFO] Spark Project Examples ............................. SKIPPED > [INFO] Spark Project REPL ................................. SKIPPED > [INFO] Spark Project YARN Parent POM ...................... SKIPPED > [INFO] Spark Project YARN Stable API ...................... SKIPPED > [INFO] Spark Project YARN Shuffle Service ................. SKIPPED > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD FAILURE > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 03:49 min > [INFO] Finished at: 2014-12-30T12:41:59+01:00 > [INFO] Final Memory: 59M/417M > [INFO] > ------------------------------------------------------------------------ > [WARNING] The requested profile "hadoop-2.5" could not be activated because > it does not exist. > [ERROR] Failed to execute goal on project spark-assembly_2.10: Could not > resolve dependencies for project > org.apache.spark:spark-assembly_2.10:pom:1.2.0: The following artifacts > could not be resolved: org.apache.spark:spark-repl_2.11:jar:1.2.0, > org.apache.spark:spark-yarn_2.11:jar:1.2.0: Could not find artifact > org.apache.spark:spark-repl_2.11:jar:1.2.0 in central > (https://repo1.maven.org/maven2) -> [Help 1] > org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute > goal on project spark-assembly_2.10: Could not resolve dependencies for > project org.apache.spark:spark-assembly_2.10:pom:1.2.0: The following > artifacts could not be resolved: org.apache.spark:spark-repl_2.11:jar:1.2.0, > org.apache.spark:spark-yarn_2.11:jar:1.2.0: Could not find artifact > org.apache.spark:spark-repl_2.11:jar:1.2.0 in central > (https://repo1.maven.org/maven2) --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
