Ian Hellstrom created SPARK-17081:
-------------------------------------
Summary: Empty strings not preserved which causes SQLException:
mismatching column value count
Key: SPARK-17081
URL: https://issues.apache.org/jira/browse/SPARK-17081
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 1.4.1
Reporter: Ian Hellstrom
When writing a DataFrame that contains empty strings as values to an RDBMS, the
query that is generated does not have the correct column count:
{code}
CREATE TABLE demo(foo INTEGER, bar VARCHAR(10));
-----
case class Record(foo: Int, bar: String)
val data = sc.parallelize(List(Record(1, ""))).toDF
data.write.mode("append").jdbc(...)
{code}
This causes:
{code}
java.sql.SQLException: Column count doesn't match value count at row 1
{code}
Proposal: leave empty strings as they are or convert these to NULL (although
that may not be what's intended by the user, so make this configurable).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
