Hi,
There is a bug in partitioners - see ticket Giraph 886. The ticket contains
patch, you can try to push it to Giraph trunk (I will buy you beer). You can
also make your own Partitioner by coping the original ones and fix the bug only
in your code.
Lukas
Hello,
When I change the Partition Balance Algorithm from static to
edges/vertices, Giraph 1.1 quickly crashes. I have checked my configuration
settings to make sure nothing else is out of place,
and the logs are below. Has anyone else gotten this feature working ? Or
have any thoughts
on the problem ?
Thanks,
Kenrick
======
Error logs
ERROR [compute-0] org.apache.giraph.utils.LogStacktraceCallable:
Execution of callable failed
java.lang.NullPointerException
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
at
org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-09-21 20:58:02,575 ERROR [main]
org.apache.giraph.graph.GraphMapper: Caught an unrecoverable exception
waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
java.lang.IllegalStateException: waitFor: ExecutionException occurred
while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
at
org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:838)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:415)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:93)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
at
org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2015-09-21 20:58:02,575 ERROR [main]
org.apache.giraph.worker.BspServiceWorker: unregisterHealth: Got
failure, unregistering health on
/_hadoopBsp/job_1442858202015_0001/_applicationAttemptsDir/0/_superstepDir/0/_workerHealthyDir/n06_0
on superstep 0
2015-09-21 20:58:02,579 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.lang.IllegalStateException: run: Caught an unrecoverable
exception waitFor: ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:104)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.lang.IllegalStateException: waitFor:
ExecutionException occurred while waiting for
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@4bb488d0
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:193)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:151)
at
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:136)
at
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:99)
at
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:233)
at
org.apache.giraph.graph.GraphTaskManager.processGraphPartitions(GraphTaskManager.java:838)
at
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:415)
at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:93)
... 7 more
Caused by: java.util.concurrent.ExecutionException:
java.lang.NullPointerException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:202)
at
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:312)
at
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:185)
... 14 more
Caused by: java.lang.NullPointerException
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:132)
at
org.apache.giraph.comm.messages.primitives.long_id.LongByteArrayMessageStore.getVertexMessages(LongByteArrayMessageStore.java:50)
at
org.apache.giraph.graph.ComputeCallable.computePartition(ComputeCallable.java:232)
at
org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:161)
at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:69)
at
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
======
