Github user rdblue commented on a diff in the pull request:
https://github.com/apache/spark/pull/22009#discussion_r208334273
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/BatchReadSupportProvider.java
---
@@ -19,18 +19,18 @@
import org.apache.spark.annotation.InterfaceStability;
import org.apache.spark.sql.sources.DataSourceRegister;
-import org.apache.spark.sql.sources.v2.reader.DataSourceReader;
+import org.apache.spark.sql.sources.v2.reader.BatchReadSupport;
import org.apache.spark.sql.types.StructType;
/**
* A mix-in interface for {@link DataSourceV2}. Data sources can implement
this interface to
- * provide data reading ability and scan the data from the data source.
+ * provide data reading ability for batch processing.
--- End diff --
I think this interface (and the continuous and micro-batch equivalents)
should note that returning a `ReadSupport` from options is for sources with no
catalog support or to use an implementation directly. Maybe we should add this
after #21306 is in though. What do you think?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
