Hello, After reading the following pages: https://spark.apache.org/docs/latest/building-with-maven.html http://spark.apache.org/docs/latest/hadoop-third-party-distributions.html
I came up with this build command: mvn -Dhadoop.version=1.0.3 -DskipTests -Pdeb -Phadoop-1.0.3 -Phive -Phadoop-provided clean package This fails with the following error: [WARNING] The requested profile "hadoop-1.0.3" could not be activated because it does not exist. [ERROR] Failed to execute goal on project spark-parent: Could not resolve dependencies for project org.apache.spark:spark-parent:pom:1.1.0: The following artifacts could not be resolved: org.apache.hadoop:hadoop-yarn-api:jar:1.0.3, org.apache.hadoop:hadoop-yarn-common:jar:1.0.3, org.apache.hadoop:hadoop-yarn-server-web-proxy:jar:1.0.3, org.apache.hadoop:hadoop-yarn-client:jar:1.0.3: Failure to find org.apache.hadoop:hadoop-yarn-api:jar:1.0.3 in https://repo1.maven.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced -> [Help 1] If I remove the hadoop-provided profile, it works fine. So my questions: - Why isn't there a hadoop-1.0.3 profile? The hadoop-third-party-distributions.html page seems to suggest that it would exist. - It's true, I don't see a hadoop-yarn-api:jar:1.0.3 in the maven repository. I'd like to build without yarn support. And ideas why it's pulling yarn? Is it due to the fact that the hadoop-1.0.3 profile doesn't exist? - Since running without the hadoop-provided profile works, is there any issue running that version of Spark against my Apache 1.0.3 cluster? Thanks, -Chris -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Problems-building-Spark-for-Hadoop-1-0-3-tp16324.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
