Github user cloud-fan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22009#discussion_r209133225
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/sources/v2/SessionConfigSupport.java
---
@@ -27,10 +27,10 @@
@InterfaceStability.Evolving
public interface SessionConfigSupport extends DataSourceV2 {
- /**
- * Key prefix of the session configs to propagate. Spark will extract
all session configs that
- * starts with `spark.datasource.$keyPrefix`, turn
`spark.datasource.$keyPrefix.xxx -> yyy`
- * into `xxx -> yyy`, and propagate them to all data source
operations in this session.
- */
- String keyPrefix();
+ /**
+ * Key prefix of the session configs to propagate. Spark will extract
all session configs that
+ * starts with `spark.datasource.$keyPrefix`, turn
`spark.datasource.$keyPrefix.xxx -> yyy`
--- End diff --
`datasource` is a string literal here. So kafka source should implement the
`keyPrefix` as `kafka`, and then all configs starts with
`spark.datasource.kafka` will be propagated to kafka source.
see the test:
https://github.com/windjammertechnologies/spark/commit/9c289a5cb46e00cd60db4794357f070dfdf80691#diff-e6ed4ac7b1ceb2f3a25e92b031aaecbbR24
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
