[
https://issues.apache.org/jira/browse/SPARK-10756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-10756.
-------------------------------
Resolution: Duplicate
> DataFrame write to teradata using jdbc not working, tries to create table
> each time irrespective of table existence
> -------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-10756
> URL: https://issues.apache.org/jira/browse/SPARK-10756
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.5.0
> Reporter: Amit
>
> DataFrame write to teradata using jdbc not working, tries to create table
> each time irrespective of table existence.
> Whenever it goes to persist dataframe it checks for the table existence by
> using query "SELECT 1 FROM $table LIMIT 1" and the keyword limit is not
> supported in teradata. So the exception thrown by teradata for keyword is
> understood as exception for table not exist in Spark and then Spark runs the
> create table command irrespective of table was present.
> So Create table command execution fails with the exception of table already
> exist hence saving of data frame fails
> Below is the method of JDBCUtils class
> /**
> * Returns true if the table already exists in the JDBC database.
> */
> def tableExists(conn: Connection, table: String): Boolean = {
> // Somewhat hacky, but there isn't a good way to identify whether a table
> exists for all
> // SQL database systems, considering "table" could also include the
> database name.
> Try(conn.prepareStatement(s"SELECT 1 FROM $table LIMIT
> 1").executeQuery().next()).isSuccess
> }
> In case of teradata, It returns false for every save/write operation
> irrespective of the fact that table was present.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
