cloud-fan commented on a change in pull request #24129: [SPARK-27190][SQL] add
table capability for streaming
URL: https://github.com/apache/spark/pull/24129#discussion_r267985760
##########
File path: sql/core/src/main/java/org/apache/spark/sql/sources/v2/Table.java
##########
@@ -27,11 +28,11 @@
* implementation can be a directory on the file system, a topic of Kafka, or
a table in the
* catalog, etc.
* <p>
- * This interface can mixin the following interfaces to support different
operations, like
- * {@code SupportsRead}.
+ * This interface can mixin {@link SupportsRead} and {@link SupportsWrite} to
provide data reading
+ * and writing ability.
*/
@Evolving
-public interface Table {
+public interface Table extends BaseStreamingSink {
Review comment:
Because `Table` is used to identify a source in `MicroBatchExecution`, to
track the progress of all the sources in a micro-batch query.
This is only a problem in `MicroBatchExecution`, because it needs to support
micro-bath source v1 (there is no continuous source v1) and ds v2 together. The
way we did it is to create a common interface as the source representation for
both micro-bath source v1 and ds v2, so that we can unify the code to track
progress of sources, see
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/ProgressReporter.scala
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
