Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/20386#discussion_r164648815
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java
---
@@ -63,32 +65,30 @@
DataWriterFactory<Row> createWriterFactory();
/**
- * Commits this writing job with a list of commit messages. The commit
messages are collected from
- * successful data writers and are produced by {@link
DataWriter#commit()}.
+ * Handles a commit message produced by {@link DataWriter#commit()}.
*
* If this method fails (by throwing an exception), this writing job is
considered to to have been
- * failed, and {@link #abort(WriterCommitMessage[])} would be called.
The state of the destination
- * is undefined and @{@link #abort(WriterCommitMessage[])} may not be
able to deal with it.
+ * failed, and {@link #abort()} would be called. The state of the
destination
+ * is undefined and @{@link #abort()} may not be able to deal with it.
--- End diff --
add some more comments to say that, implementations should probably cache
the commit messages and do the final step in #commit
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
