[ 
https://issues.apache.org/jira/browse/SPARK-23370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harleen Singh Mann updated SPARK-23370:
---------------------------------------
    Description: 
Currently, on jdbc read spark obtains the schema of a table from using 
{color:#654982} resultSet.getMetaData.getColumnType{color}

This works 99.99% of the times except when the column of Number type is added 
on an Oracle table using the alter statement. This is essentially an Oracle DB 
+ JDBC bug that has been documented on Oracle KB and patches exist. [oracle 
KB|https://support.oracle.com/knowledge/Oracle%20Database%20Products/1266785_1.html]

{color:#ff0000}As a result of the above mentioned issue, Spark receives a size 
of 0 for the field and defaults the field type to be BigDecimal(30,10) instead 
of what it actually should be. This is done in OracleDialect.scala. This may 
cause issues in the downstream application where relevant information may be 
missed to the changed precision and scale.{color}

_The versions that are affected are:_ 
 _JDBC - Version: 11.2.0.1 and later   [Release: 11.2 and later ]_
 _Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.1_  
_[Release: 11.1 to 11.2]_ 

+Proposed approach:+

There is another way of fetching the schema information in Oracle: Which is 
through the all_tab_columns table. If we use this table to fetch the precision 
and scale of Number time, the above issue is mitigated.

 

{color:#14892c}{color:#f6c342}I can implement the changes, but require some 
inputs on the approach from the gatekeepers here{color}.{color}


 {color:#14892c}PS. This is also my first Jira issue and my first fork for 
Spark, so I will need some guidance along the way. (yes, I am a newbee to this) 
Thanks...{color}

  was:
Currently, on jdbc read spark obtains the schema of a table from using 
{color:#654982} resultSet.getMetaData.getColumnType{color}

This works 99.99% of the times except when the column of Number type is added 
on an Oracle table using the alter statement. This is essentially an Oracle DB 
+ JDBC bug that has been documented on Oracle KB and patches exist. [oracle 
KB|https://support.oracle.com/knowledge/Oracle%20Database%20Products/1266785_1.html]

{color:#FF0000}As a result of the above mentioned issue, Spark receives a size 
of 0 for the field and defaults the field type to be BigDecimal(30,10) instead 
of what it actually should be. This is done in OracleDialect.scala. This may 
cause issues in the downstream application where relevant information may be 
missed to the changed precision and scale.{color}

_The versions that are affected are:_ 
_JDBC - Version: 11.2.0.1 and later   [Release: 11.2 and later ]_
_Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.1_  
_[Release: 11.1 to 11.2]_ 

+Proposed approach:+

There is another way of fetching the schema information in Oracle: Which is 
through the all_tab_columns table. If we use this table to fetch the precision 
and scale of Number time, the above issue is mitigated.

 

{color:#14892c}I can implement the changes, but require some inputs on the 
approach from the gatekeepers here.{color}
{color:#14892c}PS. This is also my first Jira issue and my first fork for 
Spark, so I will need some guidance along the way. (yes, I am a newbee to this) 
Thanks...{color}


> Spark receives a size of 0 for an Oracle Number field and defaults the field 
> type to be BigDecimal(30,10) instead of the actual precision and scale
> ---------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-23370
>                 URL: https://issues.apache.org/jira/browse/SPARK-23370
>             Project: Spark
>          Issue Type: Bug
>          Components: SQL
>    Affects Versions: 2.2.1
>         Environment: Spark 2.2
> Oracle 11g
> JDBC ojdbc6.jar
>            Reporter: Harleen Singh Mann
>            Priority: Major
>         Attachments: Oracle KB Document 1266785.pdf
>
>
> Currently, on jdbc read spark obtains the schema of a table from using 
> {color:#654982} resultSet.getMetaData.getColumnType{color}
> This works 99.99% of the times except when the column of Number type is added 
> on an Oracle table using the alter statement. This is essentially an Oracle 
> DB + JDBC bug that has been documented on Oracle KB and patches exist. 
> [oracle 
> KB|https://support.oracle.com/knowledge/Oracle%20Database%20Products/1266785_1.html]
> {color:#ff0000}As a result of the above mentioned issue, Spark receives a 
> size of 0 for the field and defaults the field type to be BigDecimal(30,10) 
> instead of what it actually should be. This is done in OracleDialect.scala. 
> This may cause issues in the downstream application where relevant 
> information may be missed to the changed precision and scale.{color}
> _The versions that are affected are:_ 
>  _JDBC - Version: 11.2.0.1 and later   [Release: 11.2 and later ]_
>  _Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.1_  
> _[Release: 11.1 to 11.2]_ 
> +Proposed approach:+
> There is another way of fetching the schema information in Oracle: Which is 
> through the all_tab_columns table. If we use this table to fetch the 
> precision and scale of Number time, the above issue is mitigated.
>  
> {color:#14892c}{color:#f6c342}I can implement the changes, but require some 
> inputs on the approach from the gatekeepers here{color}.{color}
>  {color:#14892c}PS. This is also my first Jira issue and my first fork for 
> Spark, so I will need some guidance along the way. (yes, I am a newbee to 
> this) Thanks...{color}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org

Reply via email to