Hi,
Sorry if this is a trivial question but I have been battling with it for a
while. I have a question about submitting jobs from a web application
potentially deployed on different machines than HBase. I am using
springframework and implemented a simple JobService class that acts as a spring
service. I can then implement all my jobs in separate classes and pass them to
my jobService for execution. When I run jobs through the hadoop command line,
the job shows up in the job tracker. However, when I submit jobs through my
webapp, it does not. Since I can step through the execution of my job in my
eclipse environment, I don't think this job is running in hadoop/hbase's
distributed environment. Is that right?
1. In short, how can I submit a job from my webapp and view its progress on the
jobtracker?
2. Additionally, I would also like to be able to get this information to
display
it in my own web application (so that users can track the status of their
running jobs). How can I query the Hadoop job tracker?
Thanks a lot for your help,
-Eric
here is snipets of my code:
class myJob extends IJob {
...
public Job createSubmittableJob(Configuration conf, String[] args)
Path inputDir = new Path(inputDir);
job = new Job(conf, jobName);
job.setJarByClass(Uploader.class);
FileInputFormat.setInputPaths(job, inputDir);
job.setInputFormatClass(TextInputFormat.class);
job.setMapperClass(TransactionLoader.class);
TableMapReduceUtil.initTableReducerJob(tableName, null, job);
job.setNumReduceTasks(0);
}
...
}
class myJobService {
...
public int submitJob(String name, IJob zjob, String[] args) {
job = zjob.createSubmittableJob(dsm.getHBaseConfiguration(), def, otherArgs);
// "Submit the job to the cluster and wait for it to finish."
return job.waitForCompletion(true) ? 0 : 1;
}
...
}
