[
https://issues.apache.org/jira/browse/SPARK-23370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-23370:
---------------------------------
Labels: bulk-closed (was: )
> Spark receives a size of 0 for an Oracle Number field and defaults the field
> type to be BigDecimal(30,10) instead of the actual precision and scale
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-23370
> URL: https://issues.apache.org/jira/browse/SPARK-23370
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.2.1
> Environment: Spark 2.2
> Oracle 11g
> JDBC ojdbc6.jar
> Reporter: Harleen Singh Mann
> Priority: Minor
> Labels: bulk-closed
> Attachments: Oracle KB Document 1266785.pdf
>
>
> Currently, on jdbc read spark obtains the schema of a table from using
> {color:#654982} resultSet.getMetaData.getColumnType{color}
> This works 99.99% of the times except when the column of Number type is added
> on an Oracle table using the alter statement. This is essentially an Oracle
> DB + JDBC bug that has been documented on Oracle KB and patches exist.
> [oracle
> KB|https://support.oracle.com/knowledge/Oracle%20Database%20Products/1266785_1.html]
> {color:#ff0000}As a result of the above mentioned issue, Spark receives a
> size of 0 for the field and defaults the field type to be BigDecimal(30,10)
> instead of what it actually should be. This is done in OracleDialect.scala.
> This may cause issues in the downstream application where relevant
> information may be missed to the changed precision and scale.{color}
> _The versions that are affected are:_
> _JDBC - Version: 11.2.0.1 and later [Release: 11.2 and later ]_
> _Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.1_
> _[Release: 11.1 to 11.2]_
> +Proposed approach:+
> There is another way of fetching the schema information in Oracle: Which is
> through the all_tab_columns table. If we use this table to fetch the
> precision and scale of Number time, the above issue is mitigated.
>
> {color:#14892c}{color:#f6c342}I can implement the changes, but require some
> inputs on the approach from the gatekeepers here{color}.{color}
> {color:#14892c}PS. This is also my first Jira issue and my first fork for
> Spark, so I will need some guidance along the way. (yes, I am a newbee to
> this) Thanks...{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
