Thanks for the help. This seems to do what I want, but now I'm having
another problem. I get a null pointer exception in JobClient.getFs(),
which is called by submitJob().
Is there something else I need to set in the jobConf object I pass to
submitJob()? It works fine when I use JobClient.runJob() instead of
submitJob()...
Here is the stack from when the exception is thrown:
java.lang.NullPointerException
at org.apache.hadoop.mapred.JobClient.getFs(JobClient.java:236)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:272)
at
net.intelresearch.cvmHadoop.KeyedByLocationalCode$ReduceOnlyDriver.main(Unknown
Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:71)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:143)
at net.intelresearch.cvmHadoop.Usage.main(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:155)
On 10/3/07, Arun C Murthy <[EMAIL PROTECTED]> wrote:
> On Wed, Oct 03, 2007 at 10:20:32AM -0400, Steve Schlosser wrote:
> >Hello all
> >
> >What is the procedure for reading counters after a job has run? I
> >want to be able to run my job using JobClient.runJob(), then read out
> >some counters to make a decision about whether to run another
> >reduction or exit. I can think of some hacky ways to do it, but there
> >must be some correct way to read the counters.
> >
>
> Take a look at:
> http://lucene.apache.org/hadoop/api/org/apache/hadoop/mapred/JobClient.html#submitJob(org.apache.hadoop.mapred.JobConf).
>
> That returns a RunningJob, then use
> http://lucene.apache.org/hadoop/api/org/apache/hadoop/mapred/RunningJob.html#getCounters()
>
> Arun
>
> >Thanks!
> >
> >-steve
>
