[
https://issues.apache.org/jira/browse/SPARK-11953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siva Gudavalli updated SPARK-11953:
-----------------------------------
Comment: was deleted
(was: I think we should avoid DDL.
We should have an opportunity in the DataFrameWrtier to skip the DDL.
Even though, Initially my motivation to log and talk about this Issue was
"tableExists check not working on Oracle", I believe there should be an option
where Spark do NO DDL via DataFrameWriter.
Regards
Shiv
)
> Acknowledge Append Mode in DataFrameWriter
> ------------------------------------------
>
> Key: SPARK-11953
> URL: https://issues.apache.org/jira/browse/SPARK-11953
> Project: Spark
> Issue Type: Improvement
> Components: Java API, Spark Submit, SQL
> Affects Versions: 1.4.1, 1.5.1
> Environment: Spark stand alone cluster
> Reporter: Siva Gudavalli
> Priority: Blocker
>
> DataFramewriter is not acknowledging Append Mode.
> In case of Append Mode => It is verfiying if the table exists and creating a
> new table if it is not there and then Inserting data.
> If the table exists check is failed for whatsover reason, it still assumes
> table doesnt exists and creates a new table.
> As per my understanding, Append should perform "no DDL". It is okay to throw
> error saying "table doesnt exist for Inserts'. But creating a Table should
> not be allowed in Append Mode.
> *I believe we should do two things here*
> perform DDL => if and only if table exists check results "Not exists"
> do not perform DDL => if SaveMode is Append
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
