I don't know if this is the best way, but this is how I do it.
Configuration conf = new Configuration();
JobClient jobClient = new JobClient(new
InetSocketAddress("jobTracker",9001),conf);
jobClient.setConf(conf); // Bug in constructor, doesn't set conf.
for(JobStatus js: jobClient.getAllJobs()){
// We only care about completed jobs.
if(!js.isJobComplete()){
continue;
}
// Do stuff on jobStatus.
:
:
}
You can also scrape info from http://jobtracker:50030/jobhistory.jsp
Or read it from the job's outputDir/_log/ directory.
Cheers,
Doug
On Aug 11, 2010, at 11:54 AM, Scott Whitecross wrote:
> Hi -
>
> What's the best way to list and query information on Hadoop job histories?
> For example, I'd like to see the job names from the past week against a
> Hadoop cluster I'm using. I don't see an API call or a way through the
> command line to pull the information. Is the best way writing a quick
> script to process the job history files?
>
> Thanks.
> Scott
