Why is so important to avoid having a poll? The cost is low and almost any job is running at least for a few minutes.
-- Andrei On Thu, Sep 22, 2011 at 3:07 PM, David Alves <[email protected]> wrote: > Hi Andrei > > I know… > The thing is that code used the Hadoop JobClient class's runJob() > method that actually polls for progress. > I am not using hadoop (in hindsight using the word "job" might have > been a mistake) and I was wondering if there is already a way to do that for > generic cases (e.g., scripts or java programs). > In particular as I'm collecting accurate metrics I'd like a non poll > based technique. > Even if there is none I can always try and code it, so all ideas are > welcome. > > thanks > david > > > On Sep 22, 2011, at 4:52 PM, Andrei Savu wrote: > >> This is exactly what the example code is doing (and the hadoop >> integration test). The job running code is blocking while the job is >> executing. >> >> -- Andrei Savu / andreisavu.ro >> >> On Thu, Sep 22, 2011 at 2:03 PM, David Alves <[email protected]> wrote: >>> Hi All >>> >>> I need to launch a cluster run a job and terminate the cluster as >>> the job is finished (as soon as possible). >>> Is there any "nice" way to do this, or do you have any suggestions? >>> On the top of my head I can imagine some quick and dirty solutions >>> (like creating a file whenever the task is completed and polling for its >>> existence from the whirr handler) but I'd like to do it without polling if >>> possible. Any ideas? >>> >>> thanks >>> -david >>> > >
