On 23 April 2012 22:54, Eric Baldeschwieler <[email protected]> wrote:
> > - Are the YARN APIs complete enough? I'd like to see a couple of new > frameworks implemented to work out the APIs > -- MPI - A classic with new needs > -- Maybe supporting HBase as a service within YARN? This will test APIs > -- Headless Map-Reduce or Pig jobs? One should be able to run them > without a fat client... > -- Job history tracking for arbitrary frameworks without new server code / > framework? > > -Graphing: Hama & Giraph. these would be ideal as they aren't at 1.x release stage yet, and because it makes so much sense to run them within the Hadoop cluster where the data lives. I think Giraph already sneaks in an an MR job, because the limit on Job Counters in the 1.x branch is limiting the number of supersteps. -I've ported the remote shell app to Groovy, though not actually tested it yet. What I'd like is to deploy something on every node that can take a list of nodes and a start time, and trigger an attempt to send data to the destination -that way you can qualify a cluster's bandwidth. for that I'd really like a way for any user w/ admin rights to deploy something on a named machine even if there are no free resources.
