Perfect summary! Thanks for writing it. Thanks, Roman.
On Fri, Dec 19, 2014 at 12:09 PM, Eli Reisman <[email protected]> wrote: > Giraph on YARN thus far doesn't break any compatibility with the MapReduce > version. When I was working on it more actively, it had a slightly faster > job startup but otherwise behaved similarly to the MapReduce version. > > There are a number of things design wise that could make the YARN profile > substantially better (in theory) but would require a fork or bigger design > changes/agreements about the MapReduce profiles. This would include things > like spawning the Master Giraph task in the Application Master itself, and > many other things along those lines. > > There are also a number of smaller things that would probably make a > difference like exposing YARN's per-task resource configuration features in > a more flexible way. > > I haven't had much time to hack on Giraph this past year, and at some point > last summer some folks like Muhammad Islam from LinkedIn did some great work > to update the YARN profile to run on Hadoop 2.2.0 or newer versions but > since then it hasn't gotten much love. > > I noticed there is still a note in the master POM from the original Giraph > on YARN implementation that says its compatible only with Hadoop > 2.0.3-alpha. I thought that was removed with Mohammad's Hadoop 2.2.0 patches > but apparently it wasn't. We should remove that, it's no longer accurate and > seems to be misleading people trying to build the YARN profile. > > > > On Fri, Oct 10, 2014 at 11:15 AM, Tripti Singh <[email protected]> wrote: >> >> Hi Matthew, >> I would have been thrilled to give you numbers on this one but for me the >> Application is not scaling without the out-of-core option( which isn't >> working the way it was in previous version) >> I'm still figuring it out and can get back once it's resolved. I have >> patched a few things and will share them for people who might face similar >> issue. If u have a fix for scalability, do let me know >> >> Thanks, >> Tripti >> >> Sent from my iPhone >> >> > On 06-Oct-2014, at 9:22 pm, "Matthew Cornell" <[email protected]> >> > wrote: >> > >> > Hi Folks. I don't think I paid enough attention to YARN vs. MR1 when I >> > built Giraph 1.0.0 for our system. How much better is Giraph on YARN? >> > Thank you. >> > >> > -- >> > Matthew Cornell | [email protected]
