Take a look at Oozie or Azkaban: http://www.quora.com/What-are-the-differences-advantages-disadvantages-of-Azkaban-vs-Oozie
On Wed, Jan 5, 2011 at 9:35 AM, Peter Veentjer <[email protected]> wrote: > He Guys, > > although it isn't completely related to HBase. Is there support for > scheduling map reduce jobs? > > E.g. I want to do a map reduce job that automatically removes certain > elements from hbase and perhaps some additional cleanup (I know that there > is support for lease times). > > I could have every node in the cluster schedule this job every hour, but if > there are n nodes I don't want this job to be running n times. Only 1 time > would be sufficient. I can create some checking logic for scheduling a map > reduce job where a job that should not be run is rejected/ignored, but if > there is something out of the box. > > Another question: > > We are building an environment where client provided plugins can be deployed > and obey some kind of SLA (e.g. we want it to be running on 2 nodes at > least). So the old plugin needs to be undeployed, the jars of the new plugin > need to be copied on demand to the nodes and the new plugin needs to be > deployed. Is there functionality for this behavior out of the box or does it > need to be implemented by ourselves? >
