Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/20490#discussion_r167011291
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataWriterFactory.java
---
@@ -32,6 +32,16 @@
@InterfaceStability.Evolving
public interface DataWriterFactory<T> extends Serializable {
+ /**
+ * Returns whether Spark should use the OutputCommitCoordinator to
ensure that only one attempt
+ * for each task commits.
+ *
+ * @return true if commit coordinator should be used, false otherwise.
+ */
+ default boolean useCommitCoordinator() {
--- End diff --
I moved it. Good idea.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
