I'm compiling against the latest trunk (Git SHA: 37bc2c80564b45d7e4ce95db76f5411a6b8bdb3a).
Without any changes, the command: mvn -Dmaven.test.skip=true -Phadoop_cdh4.1.2 clean install gives this error: [ERROR] Found a problem with the dependency commons-httpclient:commons-httpclient Resolved version is 3.0.1 Version 3.1 was expected by artifact: org.apache.hadoop:hadoop-common [ERROR] Found a problem with the dependency commons-logging:commons-logging Resolved version is 1.0.4 Version 1.1.1 was expected by artifacts: org.apache.hadoop:hadoop-client, org.apache.hadoop:hadoop-common [ERROR] Found a problem with the dependency org.apache.zookeeper:zookeeper Resolved version is 3.3.3 Version 3.4.3-cdh4.1.2 was expected by artifacts: org.apache.hadoop:hadoop-client, org.apache.hadoop:hadoop-common, org.apache.hadoop:hadoop-mapreduce-client-common, org.apache.hadoop:hadoop-mapreduce-client-hs, org.apache.hadoop:hadoop-mapreduce-client-jobclient, org.apache.hadoop:hadoop-mapreduce-client-shuffle It seems there's some dependency versions in Giraph's POM that conflict with the indirect dependencies of the same libraries from the hadoop version. See https://github.com/apache/giraph/blob/trunk/pom.xml#L262-L297. In particular, the versions of commons-httpclient:commons-httpclient, commons-logging:commons-logging, and org.apache.zookeeper:zookeeper seem to conflict. On Mon, Dec 2, 2013 at 5:16 PM, Ameya Vilankar <[email protected]>wrote: > Are you compiling the release branch? > Checkout the trunk branch of giraph and try compiling. It should work. > > > On Mon, Dec 2, 2013 at 5:13 PM, Manuel Lagang <[email protected]>wrote: > >> Ameya, thanks for the reply. It seems that your suggestion is to >> basically copy the 4.1.2 profile and point it to 4.2.0. I've seen similar >> suggestions, so I've tried this before, but it doesn't get around the >> compilation issues that I have. I'll have to do this anyway to get versions >> as close as possible. >> >> I've gotten Giraph to build without errors on a cdh4 maven profile, but >> I've had to disable some maven compile checks and modify some code to catch >> and rewrap some missed checked exceptions. Was this necessary, or should >> the cdh4 profile work as-is? >> >> >> On Mon, Dec 2, 2013 at 4:45 PM, Ameya Vilankar >> <[email protected]>wrote: >> >>> >>> Check out this link: >>> >>> http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/cdh4ig_topic_31.html >>> >>> You have to change the cloudera hadoop profile in your pom.xml file to >>> make it work. >>> >>> In you hadoop profile for the cloudera distribution, do the following >>> changes >>> >>> 1. Change this line(Line number 714): >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> <profile> >>> >>> <id>hadoop_cdh4.1.2</id> >>> >>> to >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> <profile> >>> >>> <id>hadoop_cdh4.4.0</id> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> 2. Change this line (Line number 716): >>> >>> >>> >>> >>> >>> >>> <properties> >>> >>> <hadoop.version>2.0.0-cdh4.1.2</hadoop.version> >>> >>> >>> >>> >>> >>> <munge.symbols>HADOOP_1_SECRET_MANAGER</munge.symbols> >>> >>> >>> >>> >>> >>> >>> to >>> >>> >>> <properties> >>> <hadoop.version>2.0.0-cdh4.4.0</hadoop.version> >>> >>> >>> >>> >>> <munge.symbols>HADOOP_1_SECRET_MANAGER</munge.symbols> >>> >>> >>> >>> >>> >>> >>> >>> >>> >>> Let me know if this works!!! >>> Thanks, >>> Ameya >>> >>> >>> >>> On Mon, Dec 2, 2013 at 12:58 PM, Manuel Lagang >>> <[email protected]>wrote: >>> >>>> Hi, >>>> >>>> I'm trying to move my giraph code to a cluster using cdh4, so I would >>>> to compile against that hadoop version. I tried using the cdh4.1.2 profile >>>> when compiling against the latest 1.1.0-SNAPSHOT >>>> (SHA: 37bc2c80564b45d7e4ce95db76f5411a6b8bdb3a) via this command: >>>> >>>> mvn -Dmaven.test.skip=true -Phadoop_cdh4.1.2 clean install >>>> >>>> I had to ignore any dependency version conflicts by changing 2 lines in >>>> the pom.xml that now look like: >>>> >>>> <failBuildInCaseOfConflict>false</failBuildInCaseOfConflict> >>>> >>>> I still get compilation errors though: >>>> >>>> [ERROR] >>>> /home/manuel/code/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[78,49] >>>> unreported exception java.lang.InterruptedException; must be caught or >>>> declared to be thrown >>>> [ERROR] >>>> /home/manuel/code/giraph/giraph-core/target/munged/main/org/apache/giraph/job/HaltApplicationUtils.java:[81,50] >>>> unreported exception java.lang.InterruptedException; must be caught or >>>> declared to be thrown >>>> >>>> Can anyone confirm if they can get the cdh4.1.2 profile to compile >>>> as-is? The actual version I want to eventually compile for is cdh4.2.0, >>>> does anyone have any tips on how to get Giraph working on that? >>>> >>>> Thanks, >>>> Manuel Lagang >>>> >>> >>> >> >
