2011/11/28 Stephen Boesch <java...@gmail.com>: > > Hi > I set up a pseudo cluster according to the instructions here > http://www.cloudera.com/blog/2011/11/building-and-deploying-mr2/. > Initially the randomwriter example worked. But after a crash on the machine > and restarting the services I am getting the errors shown below. > Jps seems to think the processes are running properly: > > had@mithril:/shared/hadoop$ jps > 7980 JobHistoryServer > 7668 NameNode > 7821 ResourceManager > 7748 DataNode > 8021 Jps > 7902 NodeManager > > $ hadoop jar hadoop-mapreduce-examples-0.23.0.jar randomwriter > -Dmapreduce.job.user.name=$USER > -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory > -Dmapreduce.randomwriter.bytespermap=10000 -Ddfs.blocksize=64m > -Ddfs.block.size=64m -libjars > $YARN_HOME/modules/hadoop-mapreduce-client-jobclient-0.23.0.jar output > > 2011-11-28 10:23:56,102 WARN conf.Configuration > (Configuration.java:set(629)) - mapred.used.genericoptionsparser is > deprecated. Instead, use mapreduce.client.genericoptionsparser.used > 2011-11-28 10:23:56,158 INFO ipc.YarnRPC (YarnRPC.java:create(47)) - > Creating YarnRPC for org.apache.hadoop.yarn.ipc.HadoopYarnProtoRPC > 2011-11-28 10:23:56,162 INFO mapred.ResourceMgrDelegate > (ResourceMgrDelegate.java:<init>(95)) - Connecting to ResourceManager at > /0.0.0.0:8040 > 2011-11-28 10:23:56,163 INFO ipc.HadoopYarnRPC > (HadoopYarnProtoRPC.java:getProxy(48)) - Creating a HadoopYarnProtoRpc proxy > for protocol interface org.apache.hadoop.yarn.api.ClientRMProtocol > 2011-11-28 10:23:56,203 INFO mapred.ResourceMgrDelegate > (ResourceMgrDelegate.java:<init>(99)) - Connected to ResourceManager at > /0.0.0.0:8040 > 2011-11-28 10:23:56,248 INFO mapreduce.Cluster > (Cluster.java:initialize(116)) - Failed to use > org.apache.hadoop.mapred.YarnClientProtocolProvider due to error: > java.lang.reflect.InvocationTargetException > 2011-11-28 10:23:56,250 INFO mapreduce.Cluster > (Cluster.java:initialize(111)) - Cannot pick > org.apache.hadoop.mapred.LocalClientProtocolProvider as the > ClientProtocolProvider - returned null protocol > 2011-11-28 10:23:56,251 INFO mapreduce.Cluster > (Cluster.java:initialize(111)) - Cannot pick > org.apache.hadoop.mapred.JobTrackerClientProtocolProvider as the > ClientProtocolProvider - returned null protocol > java.io.IOException: Cannot initialize Cluster. Please check your > configuration for mapreduce.framework.name and the correspond server > addresses. > > My *-site.xml files are precisely as shown on the instructions page. > In any case copying here the one that is most germane - mapred-site.xml > <?xml version="1.0"?> > <?xml-stylesheet href="configuration.xsl"?> > <configuration> > <property> > <name> mapreduce.framework.name</name> > <value>yarn</value> > </property> > </configuration> >
Remember that you have to configure two conf files related to Yarn, yarn-site.xml: <?xml version=”1.0″?> <configuration> <!– Site specific YARN configuration properties –> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce.shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> </configuration> and mapred-site.xml <?xml version=”1.0″?> <?xml-stylesheet href=”configuration.xsl”?> <configuration> <property> <name> mapreduce.framework.name</name> <value>yarn</value> </property> </configuration> Regards -- Marcos Luis Ortíz Valmaseda Linux Infrastructure Engineer Linux User # 418229 http://marcosluis2186.posterous.com http://www.linkedin.com/in/marcosluis2186 Twitter: @marcosluis2186
