Hello,
When I change the Partition Balance Algorithm from static to
edges/vertices, Giraph 1.1 quickly crashes. I have checked my configuration
settings to make sure nothing else is out of place,
and the logs are below. Has anyone else gotten this feature working ? Or
have any thoughts
on the problem ?
Thanks,
Kenrick
======
Error logs
ERROR [compute-0] org.apache.giraph.utils.LogStacktraceCallable:
Execution of callable failed
java.lang.NullPointerException
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
at
org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-09-21 20:58:02,575 ERROR [main]
org.apache.giraph.graph.GraphMapper: Caught an unrecoverable exception
waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
java.lang.IllegalStateException: waitFor: ExecutionException occurred
while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
at
org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:838)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:415)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:93)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
at
org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-09-21 20:58:02,575 ERROR [main]
org.apache.giraph.worker.BspServiceWorker: unregisterHealth: Got
failure, unregistering health on
/_hadoopBsp/job_1442858202015_0001/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/n06_0
on superstep 0
2015-09-21 20:58:02,579 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:104)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.IllegalStateException: waitFor:
ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
at
org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:838)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:415)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:93)
... 7 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
at
org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
======
