Hi, Devaraj k and Azuryy Yu
Thanks both of you. I just get it resolved. The problem is that below highlighted jar is not included in my java client side so that when the Job is initializing, it can not find the class YarnClientProtocolProvider to do further initialization. Then it causes the exception in Cluster.initialize() is thrown out. hadoop-mapreduce-client-jobclient-2.0.5-alpha.jar Thanks, Francis.Hu 发件人: Devaraj k [mailto:[email protected]] 发送时间: Wednesday, July 10, 2013 19:11 收件人: [email protected] 主题: RE: cannot submit a job via java client in hadoop- 2.0.5-alpha Hi Francis, Could you check whether those configuration files are getting loaded or not, There could be a chance that these configuration files are not getting loaded into configuration object due to some invalid path reason. conf1.addResource("resources/mapred-site.xml"); // Can you try printing the properties ‘yarn.resourcemanager.address’ & ‘mapreduce.framework.name’ values and check they are coming as per the config files JobConf conf = new JobConf(conf1); Thanks Devaraj k From: Azuryy Yu [mailto:[email protected]] Sent: 10 July 2013 16:22 To: [email protected] Subject: Re: cannot submit a job via java client in hadoop- 2.0.5-alpha you didn't set yarn.nodemanager.address in your yarn-site.xml On Wed, Jul 10, 2013 at 4:33 PM, Francis.Hu <[email protected]> wrote: Hi,All I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource Manager and all data nodes started and can access web ui of Resource Manager. I wrote a java client to submit a job as TestJob class below. But the job is never submitted successfully. It throws out exception all the time. My configurations are attached. Can anyone help me? Thanks. ---------my-java client public class TestJob { public void execute() { Configuration conf1 = new Configuration(); conf1.addResource("resources/core-site.xml"); conf1.addResource("resources/hdfs-site.xml"); conf1.addResource("resources/yarn-site.xml"); conf1.addResource("resources/mapred-site.xml"); JobConf conf = new JobConf(conf1); conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar"); conf.setJobName("Test"); conf.setInputFormat(TextInputFormat.class); conf.setOutputFormat(TextOutputFormat.class); conf.setOutputKeyClass(Text.class); conf.setOutputValueClass(IntWritable.class); conf.setMapperClass(DisplayRequestMapper.class); conf.setReducerClass(DisplayRequestReducer.class); FileInputFormat.setInputPaths(conf,new Path("/home/francis/hadoop-jobs/2013070907.FNODE.2.txt")); FileOutputFormat.setOutputPath(conf, new Path("/home/francis/hadoop-jobs/result/")); try { JobClient client = new JobClient(conf); RunningJob job = client.submitJob(conf); job.waitForCompletion(); } catch (IOException e) { e.printStackTrace(); } } } ----------Exception jvm 1 | java.io.IOException: Cannot initialize Cluster. Please check your configuration for mapreduce.framework.name and the correspond server addresses. jvm 1 | at org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119) jvm 1 | at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81) jvm 1 | at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74) jvm 1 | at org.apache.hadoop.mapred.JobClient.init(JobClient.java:482) jvm 1 | at org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461) jvm 1 | at com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59) Thanks, Francis.Hu
