[
https://issues.apache.org/jira/browse/SPARK-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reynold Xin updated SPARK-10170:
--------------------------------
Summary: Add DB2 JDBC dialect support (was: Writing from data frame into
db2 database using jdbc data source api fails with error for string, and
boolean column types.)
> Add DB2 JDBC dialect support
> ----------------------------
>
> Key: SPARK-10170
> URL: https://issues.apache.org/jira/browse/SPARK-10170
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.4.1, 1.5.0
> Reporter: Suresh Thalamati
> Fix For: 1.6.0
>
>
> Repro :
> -- start spark shell with classpath set to the db2 jdbc driver.
> SPARK_CLASSPATH=~/myjars/db2jcc.jar ./spark-shell
>
> // set connetion properties
> val properties = new java.util.Properties()
> properties.setProperty("user" , "user")
> properties.setProperty("password" , "password")
> // load the driver.
> Class.forName("com.ibm.db2.jcc.DB2Driver").newInstance
> // create data frame with a String type
> val empdf = sc.parallelize( Array((1,"John"), (2,"Mike"))).toDF("id", "name" )
> // write the data frame. this will fail with error.
> empdf.write.jdbc("jdbc:db2://bdvs150.svl.ibm.com:60000/SAMPLE:retrieveMessagesFromServerOnGetMessage=true;",
> "emp_data", properties)
> Error :
> com.ibm.db2.jcc.am.SqlSyntaxErrorException: TEXT
> at com.ibm.db2.jcc.am.fd.a(fd.java:679)
> at com.ibm.db2.jcc.am.fd.a(fd.java:60)
> ..
> // create data frame with String , and Boolean types
> val empdf = sc.parallelize( Array((1,"true".toBoolean ), (2,
> "false".toBoolean ))).toDF("id", "isManager")
> // write the data frame. this will fail with error.
> empdf.write.jdbc("jdbc:db2://<server>:<port>
> /SAMPLE:retrieveMessagesFromServerOnGetMessage=true;", "emp_data", properties)
> Error :
> com.ibm.db2.jcc.am.SqlSyntaxErrorException: TEXT
> at com.ibm.db2.jcc.am.fd.a(fd.java:679)
> at com.ibm.db2.jcc.am.fd.a(fd.java:60)
> Write is failing because by default JDBC data source implementation
> generating table schema with unsupported data types TEXT for String, and
> BIT1(1) for Boolean. I think String type should get mapped to CLOB/VARCHAR,
> and boolean type should be mapped to CHAR(1) for DB2 database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
