Yes, glad this was brought up, I'll post to the site wiki with my explicit apologies for not doing it sooner.
I posted the most recent "correct" way to build pure-YARN Giraph against any hadoop version on GIRAPH-688 here: https://issues.apache.org/jira/browse/GIRAPH-688 The trick is, you have to put the -D mvn option first, and set the "hadoop.version" Maven var. The versions as seen in "hadoop.version" are -not- the same as the version profiles (-P option) that we build against. So the correct format is: mvn -Dhadoop.version=2.0.3-alpha -Phadoop_yarn clean install if you want to just compile and not create our build products as in your example, replace "clean install" with "compile". -P profile should be "hadoop_yarn" for pure YARN builds, and "hadoop.version" appears as the part of the hadoop version you see on the Apache Hadoop download site, -after- the "hadoop-" part. So "hadoop-2.0.3-alpha" becomes for us "-Dhadoop.version=2.0.3-alpha" To build Giraph so that it can run on a YARN-enabled Hadoop cluster that -also- uses MapReduce v2 as the interface to the cluster (Giraph will still think its running on MapReduce, not YARN) you might do this: mvn -Phadoop_2.0.3 clean install If you want to see how we label build profiles vs. "hadoop.version" in Giraph builds, check our POM.xml files Thanks! Sorry for the confusion, this is a confusing thing since there are 2 ways to "run Giraph on YARN" as detailed above. On Wed, Jul 3, 2013 at 10:16 AM, Avery Ching <[email protected]> wrote: > Eli, any thoughts? > > > On 7/3/13 9:27 AM, Chui-Hui Chiu wrote: > >> Hello, >> >> I tried to compile the Giraph-1.1.0-SNAPSHOT for hadoop_2.0.3 or >> hadoop_yarn but all failed. >> >> The error message while the compile command is "mvn -Phadoop_yarn >> compile" is >> ===== >> [INFO] ------------------------------**------------------------------** >> ------------ >> [INFO] Reactor Summary: >> [INFO] >> [INFO] Apache Giraph Parent .............................. SUCCESS >> [1.320s] >> [INFO] Apache Giraph Core ..............................**.. FAILURE >> [12.508s] >> [INFO] Apache Giraph Examples ............................ SKIPPED >> [INFO] ------------------------------**------------------------------** >> ------------ >> [INFO] BUILD FAILURE >> [INFO] ------------------------------**------------------------------** >> ------------ >> [INFO] Total time: 14.473s >> [INFO] Finished at: Wed Jul 03 11:05:44 CDT 2013 >> [INFO] Final Memory: 14M/216M >> [INFO] ------------------------------**------------------------------** >> ------------ >> [ERROR] Failed to execute goal on project giraph-core: Could not resolve >> dependencies for project org.apache.giraph:giraph-core:**jar:1.1.0-SNAPSHOT: >> The following artifacts could not be resolved: org.apache.hadoop:hadoop-* >> *common:jar:SET_HADOOP_VERSION_**USING_MVN_DASH_D_OPTION, >> org.apache.hadoop:hadoop-**mapreduce-client-common:jar:** >> SET_HADOOP_VERSION_USING_MVN_**DASH_D_OPTION, org.apache.hadoop:hadoop-** >> mapreduce-client-core:jar:SET_**HADOOP_VERSION_USING_MVN_DASH_**D_OPTION, >> org.apache.hadoop:hadoop-yarn-**common:jar:SET_HADOOP_VERSION_**USING_MVN_DASH_D_OPTION, >> org.apache.hadoop:hadoop-yarn-**server-common:jar:SET_HADOOP_** >> VERSION_USING_MVN_DASH_D_**OPTION, org.apache.hadoop:hadoop-yarn-** >> server-resourcemanager:jar:**SET_HADOOP_VERSION_USING_MVN_**DASH_D_OPTION, >> org.apache.hadoop:hadoop-yarn-**server-nodemanager:jar:SET_** >> HADOOP_VERSION_USING_MVN_DASH_**D_OPTION, org.apache.hadoop:hadoop-yarn-* >> *server-tests:jar:tests:SET_**HADOOP_VERSION_USING_MVN_DASH_**D_OPTION: >> Could not find artifact org.apache.hadoop:hadoop-** >> common:jar:SET_HADOOP_VERSION_**USING_MVN_DASH_D_OPTION in central ( >> http://repo1.maven.org/maven2**) -> [Help 1] >> ===== >> >> >> The error message while the compile command is "mvn -Phadoop_2.0.3 >> compile" is >> ===== >> [INFO] ------------------------------**------------------------------** >> ------------ >> [INFO] Reactor Summary: >> [INFO] >> [INFO] Apache Giraph Parent .............................. SUCCESS >> [12.695s] >> [INFO] Apache Giraph Core ..............................**.. SUCCESS >> [2:10.916s] >> [INFO] Apache Giraph Examples ............................ FAILURE >> [2.286s] >> [INFO] ------------------------------**------------------------------** >> ------------ >> [INFO] BUILD FAILURE >> [INFO] ------------------------------**------------------------------** >> ------------ >> [INFO] Total time: 2:26.530s >> [INFO] Finished at: Wed Jul 03 11:03:25 CDT 2013 >> [INFO] Final Memory: 34M/348M >> [INFO] ------------------------------**------------------------------** >> ------------ >> [ERROR] Failed to execute goal on project giraph-examples: Could not >> resolve dependencies for project >> org.apache.giraph:giraph-**examples:jar:1.1.0-SNAPSHOT: >> Could not find artifact >> org.apache.giraph:giraph-core:**jar:tests:1.1.0-SNAPSHOT >> in central (http://repo1.maven.org/maven2**) -> [Help 1] >> ===== >> >> Do I miss anything? >> >> >> I also noticed that my Maven 3 download many files from the maven2 folder >> on a remote server with the following prompt. >> >> Downloading: >> http://repo1.maven.org/maven2/**org/apache/hadoop/.<http://repo1.maven.org/maven2/org/apache/hadoop/.> >> .. >> >> Is this a problem? >> >> >> Thanks, >> Chui-hui >> > >
