Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/20752#discussion_r172616601
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/streaming/StreamWriter.java
---
@@ -27,6 +28,9 @@
*
* Streaming queries are divided into intervals of data called epochs,
with a monotonically
* increasing numeric ID. This writer handles commits and aborts for each
successive epoch.
+ *
+ * Note that StreamWriter implementations should provide instances of
+ * {@link StreamingDataWriterFactory}.
--- End diff --
What about adding `createStreamWriterFactory` that returns the streaming
interface? That would make it easier for implementations and prevent throwing
cast exceptions because a `StreamingDataWriterFactory` is expected.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
