Hi, I have followed the procedure mentioned below, for building MRv2, but was not able to succeed with the task. I have got some compilation errors.
Please help me resolving this issue. [INFO] ------------------------------------------------------------------------ [INFO] Reactor Build Order: [INFO] [INFO] yarn-api [INFO] yarn-common [INFO] yarn-server-common [INFO] yarn-server-nodemanager [INFO] yarn-server-resourcemanager [INFO] yarn-server-tests [INFO] yarn-server [INFO] yarn [INFO] hadoop-mapreduce-client-core [INFO] hadoop-mapreduce-client-common [INFO] hadoop-mapreduce-client-shuffle [INFO] hadoop-mapreduce-client-app [INFO] hadoop-mapreduce-client-hs [INFO] hadoop-mapreduce-client-jobclient [INFO] hadoop-mapreduce-client [INFO] hadoop-mapreduce [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building yarn-api 1.0-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-antrun-plugin:1.6:run (create-protobuf-generated-sources-directory) @ yarn-api --- [INFO] Executing tasks main: [INFO] Executed tasks [INFO] [INFO] --- exec-maven-plugin:1.2:exec (generate-sources) @ yarn-api --- [INFO] [INFO] --- build-helper-maven-plugin:1.5:add-source (add-source) @ yarn-api --- [INFO] Source directory: D:\Code\MR-279\mapreduce\yarn\yarn-api\target\generated-sources\proto added. [INFO] [INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ yarn-api --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory D:\Code\MR-279\mapreduce\yarn\yarn-api\src\main\resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ yarn-api --- [INFO] Compiling 121 source files to D:\Code\MR-279\mapreduce\yarn\yarn-api\target\classes [INFO] ------------------------------------------------------------- [ERROR] COMPILATION ERROR : [INFO] ------------------------------------------------------------- [ERROR] \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap i\records\impl\pb\ContainerPBImpl.java:[22,33] package org.apache.commons.logging does not exist [ERROR] \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap i\records\impl\pb\ContainerPBImpl.java:[23,33] package org.apache.commons.logging does not exist [INFO] 2 errors [INFO] ------------------------------------------------------------- [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] yarn-api .......................................... FAILURE [5.610s] [INFO] yarn-common ....................................... SKIPPED [INFO] yarn-server-common ................................ SKIPPED [INFO] yarn-server-nodemanager ........................... SKIPPED [INFO] yarn-server-resourcemanager ....................... SKIPPED [INFO] yarn-server-tests ................................. SKIPPED [INFO] yarn-server ....................................... SKIPPED [INFO] yarn .............................................. SKIPPED [INFO] hadoop-mapreduce-client-core ...................... SKIPPED [INFO] hadoop-mapreduce-client-common .................... SKIPPED [INFO] hadoop-mapreduce-client-shuffle ................... SKIPPED [INFO] hadoop-mapreduce-client-app ....................... SKIPPED [INFO] hadoop-mapreduce-client-hs ........................ SKIPPED [INFO] hadoop-mapreduce-client-jobclient ................. SKIPPED [INFO] hadoop-mapreduce-client ........................... SKIPPED [INFO] hadoop-mapreduce .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 5.844s [INFO] Finished at: Thu Jul 28 16:03:07 IST 2011 [INFO] Final Memory: 9M/35M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.3.2:compile (default-compile) on project yarn-api: Compilation failure: Compilation failure: [ERROR] \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap i\records\impl\pb\ContainerPBImpl.java:[22,33] package org.apache.commons.logging does not exist [ERROR] \Code\MR-279\mapreduce\yarn\yarn-api\src\main\java\org\apache\hadoop\yarn\ap i\records\impl\pb\ContainerPBImpl.java:[23,33] package org.apache.commons.logging does not exist [ERROR] -> [Help 1] Regards, Ravi Teja -----Original Message----- From: Josh Wills [mailto:[email protected]] Sent: Saturday, July 02, 2011 6:34 AM To: [email protected] Subject: Re: trouble building MR-279 branch Hey Anupam, I think Jim is asking about building non-MR application masters using the RM/NM. I think Arun has promised to post a design doc soon. In the meantime, I'm just reading through the code and fixing little issues I come across (e.g., https://issues.apache.org/jira/browse/MAPREDUCE-2630 and https://issues.apache.org/jira/browse/MAPREDUCE-2633). Jim, what sort of app are you looking to build? I'd be happy to have some company wading through the code. Best, Josh On Fri, Jul 1, 2011 at 3:02 PM, Anupam Seth <[email protected]> wrote: > Hi Jim, > > Not sure if this helps at all, but not sure if you could get an idea by > looking at the included examples: > > > http://svn.apache.org/viewvc/hadoop/common/branches/MR-279/mapreduce/src/exa mples/org/apache/hadoop/examples > > that can be run as: > > hadoop jar $HADOOP_MAPRED_HOME/hadoop-mapred-examples-0.22.0-SNAPSHOT.jar > wordcount input output > > Regards, > Anupam > > -----Original Message----- > From: Jim Falgout [mailto:[email protected]] > Sent: Friday, July 01, 2011 4:23 PM > To: [email protected] > Subject: RE: trouble building MR-279 branch > > Thanks Anupam! I had skipped the step to get protobuf into the maven > repository. Also, I had Apache Forrest version 0.9 installed. That caused > the builds to fail on generating some docs. Installed version 0.8 and it > worked smoothly. > > On another note: do you know of any design docs for building an application > executor using the new resource manager? Just wondering how to get started? > > -----Original Message----- > From: Anupam Seth [mailto:[email protected]] > Sent: Friday, July 01, 2011 2:10 PM > To: [email protected] > Subject: RE: trouble building MR-279 branch > > Hi Jim, > > Following are some notes (thanks to Jeff Naisbitt & Bobby Evans). > > Build YARN > ========== > These are in addition to what you would normally do. > > Install protobuf > use Protobuf 2.4.0a (Download from > http://code.google.com/p/protobuf/downloads/list) I actually used 2.4.1 > and it worked > > install the protoc executable (configure, make, make install) > install the maven artifact (cd java; mvn install) > > Installing protoc requires gcc 4.1.x or higher (RHEL5). > > If the make step fails with (Valid until a fix is released for protobuf > 2.4.0a) ./google/protobuf/descriptor.h:1152: error: > `google::protobuf::internal::Mutex*google::protobuf::DescriptorPool::mutex_' > is private Replace descriptor.cc with > http://protobuf.googlecode.com/svn-history/r380/trunk/src/google/protobuf/de scriptor.cc > > Setup environment > > export MAVEN_OPTS=-Xmx512m > export LD_LIBRARY_PATH=/usr/local/lib > > Checkout the MR-279 branch > svn checkout http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279 > > Build common > > cd MR-279/common > ant veryclean mvn-install tar > > Build hdfs > > cd ../hdfs > ant veryclean mvn-install -Dresolvers=internal tar > > Build yarn > > cd ../mapreduce > mvn clean install assembly:assembly > # If tests are failing, add this option to skip them: > -Dmaven.test.skip.exec=true # NOTE: in order to build on a Mac, you must > also include the <code>-P-cbuild</code> option # (e.g. <code>mvn clean > install assembly:assembly -P-cbuild -Dmaven.test.skip.exec=true</code>) > ant veryclean jar jar-test tar -Dresolvers=internal > > -----Original Message----- > From: Jim Falgout [mailto:[email protected]] > Sent: Friday, July 01, 2011 2:00 PM > To: [email protected] > Subject: trouble building MR-279 branch > > Hi, > > I've checked out the MR-279 branch and am having trouble getting it to > build. Any docs/blogs/notes around that may give some hints? > > The build is unable to resolve the yarn-server even though I ran a > "mvn-install" in the yarn source area to install it in the local repository. > > Thanks, > Jim. > > >
