Hi,
Any thought on having config to overwrite the file with the same name
already exists (Line no 90 in org.apache.orc.impl.PhysicalFsWriter).

https://issues.apache.org/jira/browse/ORC-231

Use Case:
I am using OrcOutputFormat with MultipleOutput in my MapReduce job.
Since MultipleOutput does not uses OutputCommitter so there are
half/corrupt files are left from the failed reducers.

Here the files with the same name will be overwritten by the retry attempt
and it will guarantee correct result from a successful job.


Please suggest if any good production ready library available that can
replace MultipleOutputs and uses committer.

thanks.

Reply via email to