Github user liancheng commented on a diff in the pull request:
https://github.com/apache/spark/pull/5526#discussion_r30000532
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala ---
@@ -78,6 +83,40 @@ trait SchemaRelationProvider {
schema: StructType): BaseRelation
}
+/**
+ * ::DeveloperApi::
+ * Implemented by objects that produce relations for a specific kind of
data source
+ * with a given schema and partitioned columns. When Spark SQL is given a
DDL operation with a
+ * USING clause specified (to specify the implemented
SchemaRelationProvider), a user defined
+ * schema, and an optional list of partition columns, this interface is
used to pass in the
+ * parameters specified by a user.
+ *
+ * Users may specify the fully qualified class name of a given data
source. When that class is
+ * not found Spark SQL will append the class name `DefaultSource` to the
path, allowing for
+ * less verbose invocation. For example, 'org.apache.spark.sql.json'
would resolve to the
+ * data source 'org.apache.spark.sql.json.DefaultSource'
+ *
+ * A new instance of this class with be instantiated each time a DDL call
is made.
+ *
+ * The difference between a [[RelationProvider]] and a
[[FSBasedRelationProvider]] is
+ * that users need to provide a schema and a (possibly empty) list of
partition columns when
+ * using a SchemaRelationProvider. A relation provider can inherits both
[[RelationProvider]],
+ * and [[FSBasedRelationProvider]] if it can support schema inference,
user-specified
+ * schemas, and accessing partitioned relations.
+ */
+trait FSBasedRelationProvider {
+ /**
+ * Returns a new base relation with the given parameters, a user defined
schema, and a list of
+ * partition columns. Note: the parameters' keywords are case
insensitive and this insensitivity
+ * is enforced by the Map that is passed to the function.
+ */
+ def createRelation(
+ sqlContext: SQLContext,
+ schema: Option[StructType],
+ partitionColumns: Option[StructType],
+ parameters: Map[String, String]): BaseRelation
--- End diff --
Thanks, narrowed return type to `FSBasedRelation`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
