[
https://issues.apache.org/jira/browse/SPARK-14536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-14536.
-----------------------------
Resolution: Fixed
Assignee: Suresh Thalamati
Fix Version/s: 2.2.0
> NPE in JDBCRDD when array column contains nulls (postgresql)
> ------------------------------------------------------------
>
> Key: SPARK-14536
> URL: https://issues.apache.org/jira/browse/SPARK-14536
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.1
> Reporter: Jeremy Smith
> Assignee: Suresh Thalamati
> Labels: NullPointerException
> Fix For: 2.2.0
>
>
> At
> https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala#L453
> it is assumed that the JDBC driver will definitely return a non-null `Array`
> object from the call to `getArray`, and that in the event of a null array it
> will return an non-null `Array` object with a null underlying array. But as
> you can see here
> https://github.com/pgjdbc/pgjdbc/blob/master/pgjdbc/src/main/java/org/postgresql/jdbc/PgResultSet.java#L387
> that isn't the case, at least for PostgreSQL. This causes a
> `NullPointerException` whenever an array column contains null values. It
> seems like the PostgreSQL JDBC driver is probably doing the wrong thing, but
> even so there should be a null check in JDBCRDD. I'm happy to submit a PR if
> that would be helpful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
