Dear Wiki user, You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.
The "HowToSetupYourDevelopmentEnvironment" page has been changed by TomWhite: http://wiki.apache.org/hadoop/HowToSetupYourDevelopmentEnvironment?action=diff&rev1=29&rev2=30 Note that the {{{start-dfs.sh}}} script will not work with this set up, since it assumes that HADOOP_COMMON_HOME and HADOOP_HDFS_HOME are the same directory. + = Run MapReduce in pseudo-distributed mode from the dev tree = + + Build the packaging from the top level. This will build the distribution in an exploded format that we can run directly (i.e. no need to untar): + {{{ + mvn clean package -Pdist -DskipTests + }}} + + {{{ + export HADOOP_COMMON_HOME=$(pwd)/$(ls -d hadoop-common-project/hadoop-common/target/hadoop-common-*-SNAPSHOT) + export HADOOP_HDFS_HOME=$(pwd)/$(ls -d hadoop-hdfs-project/hadoop-hdfs/target/hadoop-hdfs-*-SNAPSHOT) + export HADOOP_MAPRED_HOME=$(pwd)/$(ls -d hadoop-mapreduce-project/target/hadoop-mapreduce-*-SNAPSHOT) + export YARN_HOME=$HADOOP_MAPRED_HOME + export PATH=$HADOOP_COMMON_HOME/bin:$HADOOP_HDFS_HOME/bin:$HADOOP_MAPRED_HOME/bin:$PATH + }}} + + Configure YARN to start a MR shuffle service: + + {{{ + cat > $YARN_HOME/conf/yarn-site.xml << EOF + <?xml version="1.0"?> + <configuration> + <!-- Site specific YARN configuration properties --> + <property> + <name>yarn.nodemanager.aux-services</name> + <value>mapreduce.shuffle</value> + </property> + <property> + <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> + <value>org.apache.hadoop.mapred.ShuffleHandler</value> + </property> + </configuration> + EOF + }}} + + Run YARN daemons: + {{{ + yarn resourcemanager + yarn nodemanager + }}} + + Run a MR job: + {{{ + cd hadoop-mapreduce-project + ant examples -Dresolvers=internal + cd .. + export HADOOP_CLASSPATH=$YARN_HOME/modules/* + mkdir in + cp BUILDING.txt in/ + hadoop jar hadoop-mapreduce-project/build/hadoop-mapreduce-examples-*.jar wordcount -Dmapreduce.job.user.name=$USER in out + }}} + = Build Errors = == /code/hadoop-core-trunk/build.xml:634: Could not create task or type of type: junit. ==
