[
https://issues.apache.org/jira/browse/SPARK-5501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14303722#comment-14303722
]
Yin Huai commented on SPARK-5501:
---------------------------------
h3. Interfaces introduced to the data source API
The PR of this JIRA introduces 1 *RelationProvider* and 1 *BaseRelation*.
{code}
trait CreateableRelationProvider {
def createRelation(
sqlContext: SQLContext,
parameters: Map[String, String],
data: DataFrame): BaseRelation
}
{code}
CreateableRelationProvider is used to create a BaseRelation from a DataFrame by
first storing the data of the DataFrame in the data source and then
instantiating a BaseRelation for the data just stored in the data source
(while, a RelationProvider and a SchemaRelationProvider are only used to
instantiate a BaseRelation for the existing data in the data source).
CreateableRelationProvider is used to support saving a DataFrame and CTAS
queries. You can mix it in with either RelationProvider or
SchemaRelationProvider (or both) to make your data source support saving a
DataFrame and CTAS queries.
{code}
trait InsertableRelation extends BaseRelation {
def insert(data: DataFrame, overwrite: Boolean): Unit
}
{code}
InsertableRelation is a kind of BaseRelation that supports inert operation. You
can mix it with other BaseRelation (e.g. TableScan) to make your relation
support INSERT INTO/OVERWRITE statements(in SQL)/operations(programmatic API).
> Write support for the data source API
> -------------------------------------
>
> Key: SPARK-5501
> URL: https://issues.apache.org/jira/browse/SPARK-5501
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Yin Huai
> Assignee: Yin Huai
> Priority: Blocker
> Fix For: 1.3.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
