Github user steveloughran commented on the issue:
https://github.com/apache/spark/pull/19269
w.r.t init, I'm thinking it's critical to get the
DataframeWriter.extraOptions down the tree. This lets committers be tuned on a
query-by-query basis for things like conflict management, rather than decide
upfront. I'm going have the Hadoop MR committer factory support a `Map[String,
String]` for the options, with the expectation that options hwere will
overrride any in the job conf.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
