Haoyuan, Did you specify HADOOP_CONF_DIR, YARN_CONF_DIR, etc. as per Step 8 of the INSTALL doc? I've been running my mrv2 cluster in pseudo-distributed mode on my linux box- happy to send you my configs if you'd like to play. There are also a couple of changes you need to make to the yarn config files to get real MR running, as Bobby alluded to in his email.
To run the example in the INSTALL file, you'll need to patch in this change: https://issues.apache.org/jira/browse/MAPREDUCE-2644 Josh On Fri, Jul 8, 2011 at 8:07 AM, Haoyuan Li <haoyuan...@gmail.com> wrote: > Hi Josh, > > Which configuration file could tell Yarn the location of HDFS? This could > be > found on > > http://svn.apache.org/repos/asf/hadoop/common/branches/MR-279/mapreduce/INSTALL > > Thank you. > > Best, > > Haoyuan > > On Fri, Jul 8, 2011 at 10:58 PM, Josh Wills <jwi...@cloudera.com> wrote: > > > Hey Bobby- > > > > Vinod and I did a cleanup pass over INSTALL and yarn/README in > > https://issues.apache.org/jira/browse/MAPREDUCE-2645. Please take a look > > and > > check if there's anything else we need to add/update. > > > > Josh > > > > On Fri, Jul 8, 2011 at 7:25 AM, Robert Evans <ev...@yahoo-inc.com> > wrote: > > > > > I mapreduce/INSTALL also has some important information in it, and be > > aware > > > that you do not have to install the avro plugin any more. Maven can > > > download it and install it automatically now, but the README was never > > > updated. Also be sure to install protocol buffers. The build will > fail > > > without it. > > > > > > --Bobby > > > > > > On 7/8/11 9:04 AM, "Josh Wills" <jwi...@cloudera.com> wrote: > > > > > > You want to generate them using mvn instead. See the > > mapreduce/yarn/README > > > file for how to do it. > > > > > > On Fri, Jul 8, 2011 at 7:00 AM, Devaraj K <devara...@huawei.com> > wrote: > > > > > > > Hi, > > > > > > > > > > > > > > > > I am getting this below errors when I try to generate eclipse files > > > using > > > > eclipse-files target. Can anybody help me? > > > > > > > > > > > > > > > > > > > > > > > > Buildfile: D:\svn\nextgenmapreduce\mapreduce\build.xml > > > > > > > > ivy-download: > > > > [get] Getting: > > > > < > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar> > > > > http://repo2.maven.org/maven2/org/apache/ivy/ivy/2.2.0/ivy-2.2.0.jar > > > > [get] To: D:\svn\nextgenmapreduce\mapreduce\ivy\ivy-2.2.0.jar > > > > [get] Not modified - so not downloaded > > > > > > > > ivy-init-dirs: > > > > > > > > ivy-probe-antlib: > > > > > > > > ivy-init-antlib: > > > > > > > > ivy-init: > > > > [ivy:configure] :: Ivy non official version - :: > > > > <http://ant.apache.org/ivy/> http://ant.apache.org/ivy/ :: > > > > [ivy:configure] :: loading settings :: file = > > > > D:\svn\nextgenmapreduce\mapreduce\ivy\ivysettings.xml > > > > > > > > ivy-resolve-common: > > > > [ivy:resolve] > > > > [ivy:resolve] :: problems summary :: > > > > [ivy:resolve] :::: WARNINGS > > > > [ivy:resolve] module not found: > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT > > > > [ivy:resolve] ==== apache-snapshot: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > oop/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > op/yarn-server-common/1.0-SNAPSHOT/yarn-server-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] ==== maven2: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP > > > > SHOT/yarn-server-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS > > > > HOT/yarn-server-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT!yarn-server-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAP > > > > SHOT/yarn-server-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-server-common/1.0-SNAPS > > > > HOT/yarn-server-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] module not found: > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT > > > > [ivy:resolve] ==== apache-snapshot: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > > > > > > > > > oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1 > > > > .0-SNAPSHOT.pom> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > > > > > > > > > > op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1. > > > > 0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > > > > > > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce > > > > -client-core.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > > > > > > > > > > > oop/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1 > > > > .0-SNAPSHOT.jar> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > > > > > > > > > > > op/hadoop-mapreduce-client-core/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1. > > > > 0-SNAPSHOT.jar > > > > [ivy:resolve] ==== maven2: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor > > > > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core > > > > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > > > > > > > > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT!hadoop-mapreduce > > > > -client-core.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-cor > > > > e/1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core > > > > /1.0-SNAPSHOT/hadoop-mapreduce-client-core-1.0-SNAPSHOT.jar > > > > [ivy:resolve] module not found: > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT > > > > [ivy:resolve] ==== apache-snapshot: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/had > > > > oop/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > https://repository.apache.org/content/repositories/snapshots/org/apache/hado > > > > op/yarn-common/1.0-SNAPSHOT/yarn-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] ==== maven2: tried > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya > > > > rn-common-1.0-SNAPSHOT.pom> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar > > > > n-common-1.0-SNAPSHOT.pom > > > > [ivy:resolve] -- artifact > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT!yarn-common.jar: > > > > [ivy:resolve] > > > > < > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/ya > > > > rn-common-1.0-SNAPSHOT.jar> > > > > > > > > > > > > > > http://repo1.maven.org/maven2/org/apache/hadoop/yarn-common/1.0-SNAPSHOT/yar > > > > n-common-1.0-SNAPSHOT.jar > > > > [ivy:resolve] > > > > :::::::::::::::::::::::::::::::::::::::::::::: > > > > > > > > [ivy:resolve] :: UNRESOLVED DEPENDENCIES > > > > :: > > > > > > > > [ivy:resolve] > > > > :::::::::::::::::::::::::::::::::::::::::::::: > > > > > > > > [ivy:resolve] :: > > > > org.apache.hadoop#yarn-server-common;1.0-SNAPSHOT: not found > > > > [ivy:resolve] :: > > > > org.apache.hadoop#hadoop-mapreduce-client-core;1.0-SNAPSHOT: not > found > > > > [ivy:resolve] :: > > > > org.apache.hadoop#yarn-common;1.0-SNAPSHOT: > > > > not found > > > > [ivy:resolve] > > > > :::::::::::::::::::::::::::::::::::::::::::::: > > > > > > > > [ivy:resolve] > > > > [ivy:resolve] > > > > [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS > > > > > > > > > > > > > > > > > > > > Devaraj K > > > > > > > > > > > > > > > > > > ---------------------------------------------------------------------------- > > > > --------------------------------------------------------- > > > > This e-mail and its attachments contain confidential information from > > > > HUAWEI, which > > > > is intended only for the person or entity whose address is listed > > above. > > > > Any > > > > use of the > > > > information contained herein in any way (including, but not limited > to, > > > > total or partial > > > > disclosure, reproduction, or dissemination) by persons other than the > > > > intended > > > > recipient(s) is prohibited. If you receive this e-mail in error, > please > > > > notify the sender by > > > > phone or email immediately and delete it!ss > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
