[
https://issues.apache.org/jira/browse/SPARK-11953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15026515#comment-15026515
]
Sean Owen commented on SPARK-11953:
-----------------------------------
Like Huaxin, I'm not sure there is a big different here. You can't append into
a non-existent table, so you're going to get an error either way. Is your point
about the particular error? or that you think the DDL should be avoided as an
optimization? Because I think your motivation is that the DDL doesn't work on
your DB, but that's a separate existing JIRA as we already discussed.
> Acknowledge Append Mode in DataFrameWriter
> ------------------------------------------
>
> Key: SPARK-11953
> URL: https://issues.apache.org/jira/browse/SPARK-11953
> Project: Spark
> Issue Type: Improvement
> Components: Java API, Spark Submit, SQL
> Affects Versions: 1.4.1, 1.5.1
> Environment: Spark stand alone cluster
> Reporter: Siva Gudavalli
> Priority: Blocker
>
> DataFramewriter is not acknowledging Append Mode.
> In case of Append Mode => It is verfiying if the table exists and creating a
> new table if it is not there and then Inserting data.
> If the table exists check is failed for whatsover reason, it still assumes
> table doesnt exists and creates a new table.
> As per my understanding, Append should perform "no DDL". It is okay to throw
> error saying "table doesnt exist for Inserts'. But creating a Table should
> not be allowed in Append Mode.
> *I believe we should do two things here*
> perform DDL => if and only if table exists check results "Not exists"
> do not perform DDL => if SaveMode is Append
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
