I agree with Eric and Rafal's assessment. You can send a patch about this. Regards, Mohammad
On Friday, January 3, 2014 1:13 PM, Rafal Wojdyla <[email protected]> wrote: I had the same problem, and I have created this ticket: https://issues.apache.org/jira/browse/GIRAPH-819 First I thought it's just operator problem, then I got to the same conclusion as you, and removed the if check. Btw - what was the purpose of that check, in the same function we check whether there is enough memory available. Memory check still doesn't guaranty that the job will run though, imagine 10 really small node manager (512MB) in total 5120MB, and giraph application that requires 2 containers 1GB each. On Fri, Jan 3, 2014 at 9:47 PM, Eric Kimbrel <[email protected]> wrote: I am running on hadoop 2.2.0-cdh5.0.0-beta-1 with pure yarn mode. I = >have noticed two issues in org.apache.giraph.yarn.GiraphYarnClient > >giraph code base is 1.1.0-SNAPSHOT, downloaded on Jan 2, 2014 > >looking at the checkPerNodeResourcesAvailable Method. > >The first issue is that the number of containers available is calculated = >using node.getNumContainers(). Looking at the yarn documentation this = >is the number of containers currently running on the node. So with a = >yarn cluster with no jobs running all nodes report 0 containers. > > >The second issue (in the same method) is this if block: > >if (workers < numContainers) { > throw new RuntimeException("Giraph job requires " + workers + > " containers to run; cluster only hosts " + numContainers); > >} > >So with the current set up if a cluster has 4 containers currently = >running and a graph job is submitted that requires 2 containers the job = >will fail saying =93Giraph job requires 2 containers to run; cluster = >only hosts 4=94. > >the if statement should be =93 workers > numContainers=94 and = >numContainers should reflect the total number of containers available, = >not the number of containers currently running. I don=92t know yarn = >well so i don=92t know if such a number is available at all. > >For the time being i plan on getting this working by bypassing the check = >all together. >
