Thomas,
Thanks for the response.
I completed the following steps.
- got the code from MR-279 branch.
- installed ant/maven.
- Got the protobuf code, compiled and installed it.
Do I need the avro-maven-plugin? When I ran the below command got the
error that the pom file was not found. Where do I get the jar and the
pom files for the avro-maven-plugin? I was able to get the source code
for them, but not the binaries.
mvn install:install-file
-Dfile=./avro-maven-plugin/avro-maven-plugin-1.4.0-SNAPSHOT.jar
-DpomFile=./avro-maven-plugin/avro-maven-plugin-1.4.0-SNAPSHOT.pom
Regards,
Praveen
On Wednesday 15 June 2011 08:19 PM, Thomas Graves wrote:
On 6/15/11 8:54 AM, "Praveen Sripati"<praveensrip...@gmail.com> wrote:
Hi,
I am trying to build and deploy MRv2 and following the instructions in the
INSTALL file. The instructions have to be modified after the recent
re-organisation (HADOOP-7106)
http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTAL>
L
1. Is the code under MR-279 branch not sufficient? Should I get the common
code from the yahoo-merge branch and the HDFS code from the HDFS-1052 branch
based on the instructions - what's the dependency?
I'll update the INSTALL doc.
You should be able to checkout just the MR-279 branch now. It pulls in the
yahoo-merge version of common and hdfs. Todd set that up to work
automatically when unsplitting, from his email:
MR-279: http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279
(this one has the yahoo-merge common and hdfs, and the MR-279 mapred)
Checkout code:
svn co http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/
-install protobuf - see here for instructions on that, I'm actually using
version 2.4.1:
http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/yarn
/README
Setup env:
- export MAVEN_OPTS=-Xmx512m
- export LD_LIBRARY_PATH=/usr/local/lib
build common
- cd common
- ant veryclean mvn-install tar
build hdfs
- cd hdfs
- ant veryclean mvn-install -Dresolvers=internal tar
build yarn
- cd mapreduce
- mvn clean install assembly:assembly
# If tests are failing, add this option to skip them:
-Dmaven.test.skip.exec=true
- ant veryclean jar jar-test tar -Dresolvers=internal
Let me know if you have any problems.
2. Should I get the rest of the code (excluding common and HDFS) from the
MR-279 branch?
Yes, see above.
3. What is the yahoo-merge branch?
yahoo-merge branch was used when yahoo merged everything back to apache
from its private branch. I believe it should be same as trunk for common
and hdfs at this point.
Tom
