First there is the question of accuracy, as I said I am collecting metrics that I'd like to be as accurate as possible. Second there is the matter of elegance. I always like to avoid polls whenever possible.
That being said, I don't wan't to embark in some odyssey just to avoid poll, so if it really is too much trouble I am ok with letting it go. Anyhow even with poll is there something already implemented that enables it in generic cases? thanks -david On Sep 22, 2011, at 5:09 PM, Andrei Savu wrote: > Why is so important to avoid having a poll? The cost is low and almost > any job is running at least for a few minutes. > > -- Andrei > > On Thu, Sep 22, 2011 at 3:07 PM, David Alves <[email protected]> wrote: >> Hi Andrei >> >> I know… >> The thing is that code used the Hadoop JobClient class's runJob() >> method that actually polls for progress. >> I am not using hadoop (in hindsight using the word "job" might have >> been a mistake) and I was wondering if there is already a way to do that for >> generic cases (e.g., scripts or java programs). >> In particular as I'm collecting accurate metrics I'd like a non poll >> based technique. >> Even if there is none I can always try and code it, so all ideas are >> welcome. >> >> thanks >> david >> >> >> On Sep 22, 2011, at 4:52 PM, Andrei Savu wrote: >> >>> This is exactly what the example code is doing (and the hadoop >>> integration test). The job running code is blocking while the job is >>> executing. >>> >>> -- Andrei Savu / andreisavu.ro >>> >>> On Thu, Sep 22, 2011 at 2:03 PM, David Alves <[email protected]> wrote: >>>> Hi All >>>> >>>> I need to launch a cluster run a job and terminate the cluster as >>>> the job is finished (as soon as possible). >>>> Is there any "nice" way to do this, or do you have any suggestions? >>>> On the top of my head I can imagine some quick and dirty solutions >>>> (like creating a file whenever the task is completed and polling for its >>>> existence from the whirr handler) but I'd like to do it without polling if >>>> possible. Any ideas? >>>> >>>> thanks >>>> -david >>>> >> >>
