Hello Sebastian, Stumbled upon the same exception the other day. Check the following JIRA issue and try the applied patch. It appears to work correctly on my cluster.
https://issues.apache.org/jira/browse/GIRAPH-788 Cheers, Alex On Monday, February 10, 2014 06:42:42 PM Sebastian Schelter wrote: > Hi, > > I'm trying to process a huge graph on our cluster and therefore enabled > out-of-core. Unfortunately, I run into a NullPointerException during the > input phase. > > What can I do to fix this? I'm using the following settings: > > giraph.oneToAllMsgSending=true > giraph.isStaticGraph=true > giraph.numComputeThreads=15 > giraph.numInputThreads=15 > giraph.numOutputThreads=15 > giraph.maxNumberOfSupersteps=30 > giraph.useOutOfCoreGraph=true > > Here's the log for the exception: > > > java.lang.IllegalStateException: run: Caught an unrecoverable exception > waitFor: ExecutionException occurred while waiting for > org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6e172129 > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101) > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.ja > va:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.IllegalStateException: waitFor: ExecutionException > occurred while waiting for > org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6e172129 > at > org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181 > ) at > org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java > :139) at > org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java > :124) at > org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils. > java:87) at > org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(Progressa > bleUtils.java:221) at > org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.j > ava:281) at > org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java > :325) at > org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:506) > at > org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:244) > at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91) > ... 7 more > Caused by: java.util.concurrent.ExecutionException: > java.lang.IllegalStateException: getOrCreatePartition: cannot retrieve > partition 14 > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262) > at java.util.concurrent.FutureTask.get(FutureTask.java:119) > at > org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(Progressabl > eUtils.java:300) at > org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173 > ) ... 16 more > Caused by: java.lang.IllegalStateException: getOrCreatePartition: cannot > retrieve partition 14 > at > org.apache.giraph.partition.DiskBackedPartitionStore.getOrCreatePartition(Di > skBackedPartitionStore.java:243) at > org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWork > erVerticesRequest.java:110) at > org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(Net > tyWorkerClientRequestProcessor.java:482) at > org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.sendVertexReq > uest(NettyWorkerClientRequestProcessor.java:276) at > org.apache.giraph.worker.VertexInputSplitsCallable.readInputSplit(VertexInpu > tSplitsCallable.java:172) at > org.apache.giraph.worker.InputSplitsCallable.loadInputSplit(InputSplitsCalla > ble.java:267) at > org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:2 > 11) at > org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:6 > 0) at > org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.jav > a:51) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:11 > 10) at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:6 > 03) at java.lang.Thread.run(Thread.java:722) > Caused by: java.util.concurrent.ExecutionException: > java.lang.NullPointerException > at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) > at java.util.concurrent.FutureTask.get(FutureTask.java:111) > at > org.apache.giraph.partition.DiskBackedPartitionStore.getOrCreatePartition(Di > skBackedPartitionStore.java:228) ... 13 more > Caused by: java.lang.NullPointerException > at > org.apache.giraph.partition.DiskBackedPartitionStore$GetPartition.call(DiskB > ackedPartitionStore.java:692) at > org.apache.giraph.partition.DiskBackedPartitionStore$GetPartition.call(DiskB > ackedPartitionStore.java:658) at > java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at > java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > org.apache.giraph.partition.DiskBackedPartitionStore$DirectExecutorService.e > xecute(DiskBackedPartitionStore.java:972) at > java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService. > java:132) ... 14 more
