OK, solved this. I am running on EC2 and did not think to explicitly open the ResourceManager port in the security group, and I also forgot to set the yarn resource manager hostname, so the nodes could not connect back to the resourcemanager. Fixed those two things, and now Giraph finds nodes, and at least tries to run my job. The problem now is that the job seems to just run forever. :-(
Phil This message optimized for indexing by NSA PRISM On Tue, Mar 10, 2015 at 9:56 PM, Phillip Rhodes <[email protected]> wrote: > Giraph Gurus: > > I'm running Giraph using Hadoop 2.5.2 on a 5 node cluster. But when I > try to run > the SimpleShortestPathsComputation example, I get this error: > > Exception in thread "main" java.lang.IllegalStateException: Giraph's > estimated cluster heap 2000MB ask is greater than the current > available cluster heap of 0MB. Aborting Job. > > So far I've been unable to determine why Giraph thinks the cluster has > a 0MB heap. I've set YARN_HEAPSIZE and HADOOP_HEAPSIZE in yarn-env.sh > and hadoop-env.sh respectively, and in yarn-site.xml I have: > > <property> > <name>yarn.nodemanager.resource.memory-mb</name> > <value>8960</value> > </property> > <property> > <name>yarn.scheduler.minimum-allocation-mb</name> > <value>2048</value> > </property> > > > Any idea what is going on with this? > > > Thanks, > > > Phil > --- > This message optimized for indexing by NSA PRISM
