[
https://issues.apache.org/jira/browse/SPARK-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shiv Prashant Sood reopened SPARK-28152:
----------------------------------------
Reopening this issue to submit this change as a separate PR for clarity.
Earlier this change for made part of the ByteType PR ( 28151)
> ShortType and FloatTypes are not correctly mapped to right JDBC types when
> using JDBC connector
> -----------------------------------------------------------------------------------------------
>
> Key: SPARK-28152
> URL: https://issues.apache.org/jira/browse/SPARK-28152
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0, 2.4.3
> Reporter: Shiv Prashant Sood
> Priority: Minor
>
> ShortType and FloatTypes are not correctly mapped to right JDBC types when
> using JDBC connector. This results in tables and spark data frame being
> created with unintended types.
> Some example issue
> * Write from df with column type results in a SQL table of with column type
> as INTEGER as opposed to SMALLINT. Thus a larger table that expected.
> * read results in a dataframe with type INTEGER as opposed to ShortType
> FloatTypes have a issue with read path. In the write path Spark data type
> 'FloatType' is correctly mapped to JDBC equivalent data type 'Real'. But in
> the read path when JDBC data types need to be converted to Catalyst data
> types ( getCatalystType) 'Real' gets incorrectly gets mapped to 'DoubleType'
> rather than 'FloatType'.
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
