[
https://issues.apache.org/jira/browse/SPARK-28151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16872577#comment-16872577
]
Shiv Prashant Sood commented on SPARK-28151:
--------------------------------------------
Fixed by [https://github.com/apache/spark/pull/24969]
> ByteType is not correctly mapped for read/write of SQLServer tables
> -------------------------------------------------------------------
>
> Key: SPARK-28151
> URL: https://issues.apache.org/jira/browse/SPARK-28151
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0, 2.4.3
> Reporter: Shiv Prashant Sood
> Priority: Minor
>
> Writing dataframe with column type BYTETYPE fails when using JDBC connector
> for SQL Server. Append and Read of tables also fail. The problem is due
> 1. (Write path) Incorrect mapping of BYTETYPE in getCommonJDBCType() in
> jdbcutils.scala where BYTETYPE gets mapped to BYTE text. It should be mapped
> to TINYINT
> {color:#cc7832}case {color}ByteType =>
> Option(JdbcType({color:#6a8759}"BYTE"{color}{color:#cc7832},
> {color}java.sql.Types.{color:#9876aa}TINYINT{color}))
> In getCatalystType() ( JDBC to Catalyst type mapping) TINYINT is mapped to
> INTEGER, while it should be mapped to BYTETYPE. Mapping to integer is ok from
> the point of view of upcasting, but will lead to 4 byte allocation rather
> than 1 byte for BYTETYPE.
> 2. (read path) Read path ends up calling makeGetter(dt: DataType, metadata:
> Metadata). The function sets the value in RDD row. The value is set per the
> data type. Here there is no mapping for BYTETYPE and thus results will result
> in an error when getCatalystType() is fixed.
> Note : These issues were found when reading/writing with SQLServer. Will be
> submitting a PR soon to fix these mappings in MSSQLServerDialect.
> Error seen when writing table
> (JDBC Write failed,com.microsoft.sqlserver.jdbc.SQLServerException: Column,
> parameter, or variable #2: *Cannot find data type BYTE*.)
> com.microsoft.sqlserver.jdbc.SQLServerException: Column, parameter, or
> variable #2: Cannot find data type BYTE.
> com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:254)
> com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1608)
> com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:859)
> ..
>
>
>
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
