Giraph really is fairly close to what we need and it is conceivable that we could do something FlumeJava-ish on top of that.
YARN is supposed to be ready for prime time in the Q1-Q2 time frame next year. When it is ready, it will be available from essentially all the Hadoop-compatible vendors. On Mon, Sep 5, 2011 at 8:02 AM, Jake Mannix <[email protected]> wrote: > Giraph is closer to our dependency-set: > a) runs on raw Hadoop 0.20.3 and 0.20.203 > b) in java > c) and is very similar to the usual programming model of M/R > > Point c) is pretty important: while you do get everything in-memory, you > still > program it in the way that is familiar to Hadoop-people: you have Job > objects you > can configure to fit your cluster's special shape and characteristics, etc. > > Anything YARN-based kinda worries me. I am not sure how soon people > will really see production-grade next-gen M/R environments available to > them. >
