Hi,

I am facing the following exception while using Delta iteration. The
dataset size is 27GB. Any suggestions to overcome this issue?

15:14:30,431 ERROR
eu.stratosphere.pact.runtime.task.RegularPactTask             - Error in
task code:  SolutionSet Delta (142/284)
java.lang.RuntimeException: Memory ran out. Compaction failed.
numPartitions: 32 minPartition: 23 maxPartition: 40 number of overflow
segments: 15 bucketSize: 242 Message: Index: 59, Size: 24
        at
eu.stratosphere.pact.runtime.hash.CompactingHashTable.insertOrReplaceRecord(CompactingHashTable.java:526)
        at
eu.stratosphere.pact.runtime.iterative.io.SolutionSetUpdateOutputCollector.collect(SolutionSetUpdateOutputCollector.java:55)
        at
eu.stratosphere.pact.runtime.task.NoOpDriver.run(NoOpDriver.java:64)
        at
eu.stratosphere.pact.runtime.task.RegularPactTask.run(RegularPactTask.java:505)
        at
eu.stratosphere.pact.runtime.iterative.task.AbstractIterativePactTask.run(AbstractIterativePactTask.java:132)
        at
eu.stratosphere.pact.runtime.iterative.task.IterationTailPactTask.run(IterationTailPactTask.java:100)
        at
eu.stratosphere.pact.runtime.task.RegularPactTask.invoke(RegularPactTask.java:370)
        at
eu.stratosphere.nephele.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:284)
        at java.lang.Thread.run(Thread.java:853)

Thanks,
Janani

Reply via email to