We could make a change of that nature. We currently use
job.waitForCompletion() to submit jobs, if we switch to job.submit() is
there an alternate blocking call to wait for the job to finish other
than polling job.isComplete()? I'm not currently seeing anything in the
API.
- Adam
On 5/10/11 3:47 PM, Aaron Baff wrote:
As part of the job submission, once it's submitted, grab the JobID from that
object and print it out on STDOUT or to a file and have your startup script(s)
parse it out from there.
--Aaron
-----Original Message-----
From: Adam Phelps [mailto:[email protected]]
Sent: Tuesday, May 10, 2011 3:45 PM
To: [email protected]
Subject: Getting (or setting) a job ID
We have intermittently seen cases where a job will "freeze" for some as
yet unknown reason, and thereby block other processes waiting for that
job to complete. I'm trying to modify our job-launching scripts to kill
the job if it doesn't complete in a reasonable amount of time, however
to do that I would need to know the job ID (ie along the lines of
job_201104292017_9008).
As far as I can tell there is no good way to map the name we use to
create the job to the job ID from the command line (none of the "hadoop
job" options do it). Am I missing something in the CLI that would help
with this? Alternate ideas are also welcome.
- Adam
