Hi Sebastian, I would solve this within the worker, which takes the number of the superstep to create a snapshot in an individual subdirectory to store a snapshot of the graph which consists of all individual partitions. Finally you just merge all partition files to one graph representation. It is not the same thing like using an OutpuFormat, I think. The subdirectorys should be created at the beginn of the super step before the workers start processing the data. I did not implement this until now, but I work on the same or even a comparable requirement.
Best wishes Mirko On Sat, Jan 18, 2014 at 10:02 AM, Sebastian Schelter <[email protected]> wrote: > Hi, > > Did we have a way to write out the state of the graph after each > superstep? I have an algorithm that requires this and I don't want to > buffer the intermediate results in memory until the algorithm finishes. > > --sebastian > --
