[ https://issues.apache.org/jira/browse/SPARK-21740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16375134#comment-16375134 ]
Dheeren Beborrtha commented on SPARK-21740: ------------------------------------------- What workaround did you use? DId you modify Spark-core or modify phoenix jdbc client? > DataFrame.write does not work with Phoenix JDBC Driver > ------------------------------------------------------ > > Key: SPARK-21740 > URL: https://issues.apache.org/jira/browse/SPARK-21740 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.0.0, 2.2.0 > Reporter: Paul Wu > Priority: Major > Labels: jdbc, phoenix > > The reason for this is that Phoenix JDBC driver does not support "INSERT", > but "UPSERT". > Exception for the following program: > 17/08/15 12:18:53 ERROR Executor: Exception in task 0.0 in stage 1.0 (TID 1) > org.apache.phoenix.exception.PhoenixParserException: ERROR 601 (42P00): > Syntax error. Encountered "INSERT" at line 1, column 1. > at > org.apache.phoenix.exception.PhoenixParserException.newException(PhoenixParserException.java:33) > {code:java} > public class HbaseJDBCSpark { > private static final SparkSession sparkSession > = SparkSession.builder() > .config("spark.sql.warehouse.dir", "file:///temp") > .config("spark.driver.memory", "5g") > .master("local[*]").appName("Spark2JdbcDs").getOrCreate(); > static final String JDBC_URL > = "jdbc:phoenix:somehost:2181:/hbase-unsecure"; > public static void main(String[] args) { > final Properties connectionProperties = new Properties(); > Dataset<Row> jdbcDF > = sparkSession.read() > .jdbc(JDBC_URL, "javatest", connectionProperties); > jdbcDF.show(); > String url = JDBC_URL; > Properties p = new Properties(); > p.put("driver", "org.apache.phoenix.jdbc.PhoenixDriver"); > //p.put("batchsize", "100000"); > jdbcDF.write().mode(SaveMode.Append).jdbc(url, "javatest", p); > sparkSession.close(); > } > // Create variables > } > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org