[
https://issues.apache.org/jira/browse/SPARK-14525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15255136#comment-15255136
]
Justin Pihony commented on SPARK-14525:
---------------------------------------
If I am to update the jdbc.DefaultSource to be a CreatableRelationProvider,
then that also means that I have to update it to a SchemaRelationProvider. This
would require a change to the JDBCRelation class so that it can optionally
accept a user-specified schema. This is all possible and I see it as a change
from either:
{code}
override val schema: StructType = JDBCRDD.resolveTable(url, table, properties)
{code}
To:
{code}
override val schema: StructType = {
val resolvedSchema = JDBCRDD.resolveTable(url, table, properties)
providedSchemaOption match {
case Some(providedSchema) => {
if(providedSchema == resolvedSchema) resolvedSchema
else sys.error("User specified schema does not match the actual schema")
}
case None => resolvedSchema
}
}
{code}
Or, do the checking on initialization, which would not be lazy.
Thoughts/Preferences? Should I just skip making it a CreatableRelationProvider
if none of the above work?
> DataFrameWriter's save method should delegate to jdbc for jdbc datasource
> -------------------------------------------------------------------------
>
> Key: SPARK-14525
> URL: https://issues.apache.org/jira/browse/SPARK-14525
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.6.1
> Reporter: Justin Pihony
> Priority: Minor
>
> If you call {code}df.write.format("jdbc")...save(){code} then you get an
> error
> bq. org.apache.spark.sql.execution.datasources.jdbc.DefaultSource does not
> allow create table as select
> save is a more intuitive guess on the appropriate method to call, so the user
> should not be punished for not knowing about the jdbc method.
> Obviously, this will require the caller to have set up the correct parameters
> for jdbc to work :)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
