Exception with out-of-core enabled

Mon, 10 Feb 2014 09:47:20 -0800 

Hi,
I'm trying to process a huge graph on our cluster and therefore enabled out-of-core. Unfortunately, I run into a NullPointerException during the input phase. 

What can I do to fix this? I'm using the following settings:

giraph.oneToAllMsgSending=true
giraph.isStaticGraph=true
giraph.numComputeThreads=15
giraph.numInputThreads=15
giraph.numOutputThreads=15
giraph.maxNumberOfSupersteps=30
giraph.useOutOfCoreGraph=true

Here's the log for the exception:



java.lang.IllegalStateException: run: Caught an unrecoverable exception 
waitFor: ExecutionException occurred while waiting for 
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6e172129

        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:101)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:415)

	at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)

        at org.apache.hadoop.mapred.Child.main(Child.java:249)

Caused by: java.lang.IllegalStateException: waitFor: ExecutionException 
occurred while waiting for 
org.apache.giraph.utils.ProgressableUtils$FutureWaitable@6e172129
	at 
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:181)
	at 
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:139)
	at 
org.apache.giraph.utils.ProgressableUtils.waitForever(ProgressableUtils.java:124)
	at 
org.apache.giraph.utils.ProgressableUtils.getFutureResult(ProgressableUtils.java:87)
	at 
org.apache.giraph.utils.ProgressableUtils.getResultsWithNCallables(ProgressableUtils.java:221)
	at 
org.apache.giraph.worker.BspServiceWorker.loadInputSplits(BspServiceWorker.java:281)
	at 
org.apache.giraph.worker.BspServiceWorker.loadVertices(BspServiceWorker.java:325)
	at 
org.apache.giraph.worker.BspServiceWorker.setup(BspServiceWorker.java:506)
	at 
org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:244)

        at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:91)
        ... 7 more

Caused by: java.util.concurrent.ExecutionException: 
java.lang.IllegalStateException: getOrCreatePartition: cannot retrieve 
partition 14

        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:262)
        at java.util.concurrent.FutureTask.get(FutureTask.java:119)

	at 
org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:300)
	at 
org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:173)

        ... 16 more

Caused by: java.lang.IllegalStateException: getOrCreatePartition: cannot 
retrieve partition 14
	at 
org.apache.giraph.partition.DiskBackedPartitionStore.getOrCreatePartition(DiskBackedPartitionStore.java:243)
	at 
org.apache.giraph.comm.requests.SendWorkerVerticesRequest.doRequest(SendWorkerVerticesRequest.java:110)
	at 
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:482)
	at 
org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.sendVertexRequest(NettyWorkerClientRequestProcessor.java:276)
	at 
org.apache.giraph.worker.VertexInputSplitsCallable.readInputSplit(VertexInputSplitsCallable.java:172)
	at 
org.apache.giraph.worker.InputSplitsCallable.loadInputSplit(InputSplitsCallable.java:267)
	at 
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:211)
	at 
org.apache.giraph.worker.InputSplitsCallable.call(InputSplitsCallable.java:60)
	at 
org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)

	at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
	at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)

        at java.lang.Thread.run(Thread.java:722)

Caused by: java.util.concurrent.ExecutionException: 
java.lang.NullPointerException

        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
        at java.util.concurrent.FutureTask.get(FutureTask.java:111)

	at 
org.apache.giraph.partition.DiskBackedPartitionStore.getOrCreatePartition(DiskBackedPartitionStore.java:228)

        ... 13 more
Caused by: java.lang.NullPointerException

	at 
org.apache.giraph.partition.DiskBackedPartitionStore$GetPartition.call(DiskBackedPartitionStore.java:692)
	at 
org.apache.giraph.partition.DiskBackedPartitionStore$GetPartition.call(DiskBackedPartitionStore.java:658)

        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
        at java.util.concurrent.FutureTask.run(FutureTask.java:166)

	at 
org.apache.giraph.partition.DiskBackedPartitionStore$DirectExecutorService.execute(DiskBackedPartitionStore.java:972)
	at 
java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:132)

        ... 14 more






















					Reply via email to