There are some methods in Vertex (i.e. preApplication(), preSuperstep(),
postApplication(), postSuperstep()) that can be overidden to do anything
you like, for instance write out some data to an HDFS file. We have an
open issue on outputting Aggregator values that is unassigned if you'd
like to take a look at it as well
Hope this helps,
On 9/22/11 7:34 AM, Claudio Martella wrote:
I have the need to emit to HDFS once in a while some Text. This
doesn't happen necessarily at the end of the computation and I might
need to emit something more complex than just the VertexValue, so I'd
like more control than what the VertexWriter gives me.
What do you suggest I might do to obtain a handler to a HDFS file (it
can be in parts aswell) to write to?
Is there any code I can start looking at?