HeartSaVioR commented on a change in pull request #29767:
URL: https://github.com/apache/spark/pull/29767#discussion_r500194696
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala
##########
@@ -457,6 +470,17 @@ final class DataStreamWriter[T] private[sql](ds:
Dataset[T]) {
foreachBatch((batchDs: Dataset[T], batchId: Long) =>
function.call(batchDs, batchId))
}
+ /**
+ * Specifies the underlying output table.
+ *
+ * @since 3.1.0
+ */
+ def table(tableName: String): DataStreamWriter[T] = {
Review comment:
DataFrameWriterV2 enforces the flow perfectly (let's put aside the flow
branch for creating table), define a sink by providing table identifier,
provide options, and decide which kind of write. The flow is uni-direction and
no longer be arbitrary.
I'm feeling we should also have DataStreamWriterV2 to enforce the flow as
well, but let's do this first and have more time to think about.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
