I am using JobControl.add() to add a job and running job control in a separate thread and using JobControl.allFinished() to see all jobs completed or not . Is this work same as Job.submit()??
On Tue, Jul 26, 2011 at 4:08 PM, Harsh J <[email protected]> wrote: > Madhu, > > Do you get a specific error message / stack trace? Could you also > paste your JT logs? > > On Tue, Jul 26, 2011 at 4:05 PM, madhu phatak <[email protected]> > wrote: > > Hi > > I am using the same APIs but i am not able to run the jobs by just > adding > > the configuration files and jars . It never create a job in Hadoop , it > just > > shows cleaning up staging area and fails. > > > > On Tue, Jul 26, 2011 at 3:46 PM, Devaraj K <[email protected]> wrote: > > > >> Hi Madhu, > >> > >> You can submit the jobs using the Job API's programmatically from any > >> system. The job submission code can be written this way. > >> > >> // Create a new Job > >> Job job = new Job(new Configuration()); > >> job.setJarByClass(MyJob.class); > >> > >> // Specify various job-specific parameters > >> job.setJobName("myjob"); > >> > >> job.setInputPath(new Path("in")); > >> job.setOutputPath(new Path("out")); > >> > >> job.setMapperClass(MyJob.MyMapper.class); > >> job.setReducerClass(MyJob.MyReducer.class); > >> > >> // Submit the job > >> job.submit(); > >> > >> > >> > >> For submitting this, need to add the hadoop jar files and configuration > >> files in the class path of the application from where you want to submit > >> the > >> job. > >> > >> You can refer this docs for more info on Job API's. > >> > >> > http://hadoop.apache.org/mapreduce/docs/current/api/org/apache/hadoop/mapred > >> uce/Job.html > >> > >> > >> > >> Devaraj K > >> > >> -----Original Message----- > >> From: madhu phatak [mailto:[email protected]] > >> Sent: Tuesday, July 26, 2011 3:29 PM > >> To: [email protected] > >> Subject: Submitting and running hadoop jobs Programmatically > >> > >> Hi, > >> I am working on a open source project > >> Nectar<https://github.com/zinnia-phatak-dev/Nectar> where > >> i am trying to create the hadoop jobs depending upon the user input. I > was > >> using Java Process API to run the bin/hadoop shell script to submit the > >> jobs. But it seems not good way because the process creation model is > >> not consistent across different operating systems . Is there any better > way > >> to submit the jobs rather than invoking the shell script? I am using > >> hadoop-0.21.0 version and i am running my program in the same user where > >> hadoop is installed . Some of the older thread told if I add > configuration > >> files in path it will work fine . But i am not able to run in that way . > So > >> anyone tried this before? If So , please can you give detailed > instruction > >> how to achieve it . Advanced thanks for your help. > >> > >> Regards, > >> Madhukara Phatak > >> > >> > > > > > > -- > Harsh J >
