[ https://issues.apache.org/jira/browse/SPARK-23370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16359200#comment-16359200 ]
Harleen Singh Mann commented on SPARK-23370: -------------------------------------------- [~srowen] Yes should be able to implement in the Oracle JDBC dialect. I want to start working on it once we agree it adds value. Do you mean overhead for Spark? Or for the Oracle DB? Or for the developer? haha > Spark receives a size of 0 for an Oracle Number field and defaults the field > type to be BigDecimal(30,10) instead of the actual precision and scale > --------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: SPARK-23370 > URL: https://issues.apache.org/jira/browse/SPARK-23370 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 2.2.1 > Environment: Spark 2.2 > Oracle 11g > JDBC ojdbc6.jar > Reporter: Harleen Singh Mann > Priority: Minor > Attachments: Oracle KB Document 1266785.pdf > > > Currently, on jdbc read spark obtains the schema of a table from using > {color:#654982} resultSet.getMetaData.getColumnType{color} > This works 99.99% of the times except when the column of Number type is added > on an Oracle table using the alter statement. This is essentially an Oracle > DB + JDBC bug that has been documented on Oracle KB and patches exist. > [oracle > KB|https://support.oracle.com/knowledge/Oracle%20Database%20Products/1266785_1.html] > {color:#ff0000}As a result of the above mentioned issue, Spark receives a > size of 0 for the field and defaults the field type to be BigDecimal(30,10) > instead of what it actually should be. This is done in OracleDialect.scala. > This may cause issues in the downstream application where relevant > information may be missed to the changed precision and scale.{color} > _The versions that are affected are:_ > _JDBC - Version: 11.2.0.1 and later [Release: 11.2 and later ]_ > _Oracle Server - Enterprise Edition - Version: 11.1.0.6 to 11.2.0.1_ > _[Release: 11.1 to 11.2]_ > +Proposed approach:+ > There is another way of fetching the schema information in Oracle: Which is > through the all_tab_columns table. If we use this table to fetch the > precision and scale of Number time, the above issue is mitigated. > > {color:#14892c}{color:#f6c342}I can implement the changes, but require some > inputs on the approach from the gatekeepers here{color}.{color} > {color:#14892c}PS. This is also my first Jira issue and my first fork for > Spark, so I will need some guidance along the way. (yes, I am a newbee to > this) Thanks...{color} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org