Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19294
@szhem that null path support in `FileOutputCommitter` came with the App
Master recovery work of
[MAPREDUCE-3711](https://issues.apache.org/jira/browse/MAPREDUCE-3711); its,
trying to minimise the amount of HDFS IO done during the recovery process.
I don't think that's a codepath spark goes near; in the normal execution
paths, `FileOutputFormat` & `FileOutputCommitter` will need output paths.
(disclaimer: the more I read of that code, the less I understand it. do not
treat my opinions as normative in any way)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
