Hi Andrei
I know…
The thing is that code used the Hadoop JobClient class's runJob()
method that actually polls for progress.
I am not using hadoop (in hindsight using the word "job" might have
been a mistake) and I was wondering if there is already a way to do that for
generic cases (e.g., scripts or java programs).
In particular as I'm collecting accurate metrics I'd like a non poll
based technique.
Even if there is none I can always try and code it, so all ideas are
welcome.
thanks
david
On Sep 22, 2011, at 4:52 PM, Andrei Savu wrote:
> This is exactly what the example code is doing (and the hadoop
> integration test). The job running code is blocking while the job is
> executing.
>
> -- Andrei Savu / andreisavu.ro
>
> On Thu, Sep 22, 2011 at 2:03 PM, David Alves <[email protected]> wrote:
>> Hi All
>>
>> I need to launch a cluster run a job and terminate the cluster as the
>> job is finished (as soon as possible).
>> Is there any "nice" way to do this, or do you have any suggestions?
>> On the top of my head I can imagine some quick and dirty solutions
>> (like creating a file whenever the task is completed and polling for its
>> existence from the whirr handler) but I'd like to do it without polling if
>> possible. Any ideas?
>>
>> thanks
>> -david
>>
