Thanks Gopal for the help, If we only pass relative path in the Multioutputs. All the writer will first write to the stage location and the commit for all the writers data will be taken care by the single OutputFormat instance.
In Our code we have given absolute path instead. So the writers directly writing to the final locations and not taken care by commiter of outputFormat. I have tested the same in sample code.It is working fine. On Fri, Aug 18, 2017 at 10:45 AM, Gopal Vijayaraghavan <[email protected]> wrote: > Hi, > > > multipleOutputs.write(namedOutput, NullWritable.get(), value, path); > … > > As per my understanding Multioutputs will create multiple writers from > given outputFormat but don't use OutputCommitter. > > Does your code end up looking like this? > > https://github.com/t3rmin4t0r/tpcds-partitioner/blob/master/ > src/main/java/org/notmysock/tpcds/ParTable.java#L267 > > That's worked just fine with task failures in 2014 with ORC writers, > without an overwrite option. > > Cheers, > Gopal > > > >
