[ https://issues.apache.org/jira/browse/SPARK-10170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Reynold Xin updated SPARK-10170: -------------------------------- Summary: Add DB2 JDBC dialect support (was: Writing from data frame into db2 database using jdbc data source api fails with error for string, and boolean column types.) > Add DB2 JDBC dialect support > ---------------------------- > > Key: SPARK-10170 > URL: https://issues.apache.org/jira/browse/SPARK-10170 > Project: Spark > Issue Type: Bug > Components: SQL > Affects Versions: 1.4.1, 1.5.0 > Reporter: Suresh Thalamati > Fix For: 1.6.0 > > > Repro : > -- start spark shell with classpath set to the db2 jdbc driver. > SPARK_CLASSPATH=~/myjars/db2jcc.jar ./spark-shell > > // set connetion properties > val properties = new java.util.Properties() > properties.setProperty("user" , "user") > properties.setProperty("password" , "password") > // load the driver. > Class.forName("com.ibm.db2.jcc.DB2Driver").newInstance > // create data frame with a String type > val empdf = sc.parallelize( Array((1,"John"), (2,"Mike"))).toDF("id", "name" ) > // write the data frame. this will fail with error. > empdf.write.jdbc("jdbc:db2://bdvs150.svl.ibm.com:60000/SAMPLE:retrieveMessagesFromServerOnGetMessage=true;", > "emp_data", properties) > Error : > com.ibm.db2.jcc.am.SqlSyntaxErrorException: TEXT > at com.ibm.db2.jcc.am.fd.a(fd.java:679) > at com.ibm.db2.jcc.am.fd.a(fd.java:60) > .. > // create data frame with String , and Boolean types > val empdf = sc.parallelize( Array((1,"true".toBoolean ), (2, > "false".toBoolean ))).toDF("id", "isManager") > // write the data frame. this will fail with error. > empdf.write.jdbc("jdbc:db2://<server>:<port> > /SAMPLE:retrieveMessagesFromServerOnGetMessage=true;", "emp_data", properties) > Error : > com.ibm.db2.jcc.am.SqlSyntaxErrorException: TEXT > at com.ibm.db2.jcc.am.fd.a(fd.java:679) > at com.ibm.db2.jcc.am.fd.a(fd.java:60) > Write is failing because by default JDBC data source implementation > generating table schema with unsupported data types TEXT for String, and > BIT1(1) for Boolean. I think String type should get mapped to CLOB/VARCHAR, > and boolean type should be mapped to CHAR(1) for DB2 database. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org