jose-torres commented on a change in pull request #23619: [SPARK-26695][SQL]
data source v2 API refactor - continuous read
URL: https://github.com/apache/spark/pull/23619#discussion_r251518473
##########
File path:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/Scan.java
##########
@@ -83,4 +85,20 @@ default Batch toBatch() {
default MicroBatchStream toMicroBatchStream(String checkpointLocation) {
throw new UnsupportedOperationException("Micro-batch scans are not
supported");
}
+
+ /**
+ * Returns the physical representation of this scan for streaming query with
continuous mode. By
+ * default this method throws exception, data sources must overwrite this
method to provide an
+ * implementation, if the {@link Table} that creates this scan implements
+ * {@link SupportsContinuousRead}.
+ *
+ * @param checkpointLocation a path to Hadoop FS scratch space that can be
used for failure
+ * recovery. Data streams for the same logical
source in the same query
+ * will be given the same checkpointLocation.
+ *
+ * @throws UnsupportedOperationException
+ */
+ default ContinuousStream toContinuousStream(String checkpointLocation) {
+ throw new UnsupportedOperationException("Continuous scans are not
supported");
Review comment:
nit: I think the message should indicate they're unsupported just for this
type of Scan - this makes it sound like they're not supported in general.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
