Append is not currently supported by FileOutputFormat, so it isn't currently supported by Pig. In general appending isn't safe in MR because it is not atomic, which means there can be issues if a task fails and must be retried, with things like speculative execution, etc.
Alan. On Mar 5, 2012, at 6:27 AM, Russell Jurney wrote: > I'm curious how Pig will use append in the future? > > Russell Jurney > twitter.com/rjurney > [email protected] > datasyndrome.com > > On Mar 5, 2012, at 1:34 AM, Daan Gerits <[email protected]> wrote: > >> Hello everyone, >> >> Is Pig capable of appending to a file? I know that an exception is thrown >> when a file exists using PigStorage, but is there a way to get around this? >> >> Thanks, >> >> Daan.
