[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17237171#comment-17237171 ] Mohit Dave commented on SPARK-32587: Hi [~zdh] The issue here is if backend table has BIT datatype and we use SPARK API to read and write to the BIT type its writing NULL instead of actual value for Azure SQL DB. Steps to reproduce in Azure SQL DB : 1)Create source table with BIT datype and insert some value. 2)Create dataframe using spark.read from the source table 3)Create target table with BIT datatype. 4)Write the dataframe to Traget table using df.write API. Observation : Data is written as NULL for target BIT type.Other datatypes are working fine. > SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing > NULL values > - > > Key: SPARK-32587 > URL: https://issues.apache.org/jira/browse/SPARK-32587 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 2.4.5 >Reporter: Mohit Dave >Priority: Major > > While writing to a target in SQL Server using Microsoft's SQL Server driver > using dataframe.write API the target is storing NULL values for BIT columns. > > Table definition > Azure SQL DB > 1)Create 2 tables with column type as bit > 2)Insert some record into 1 table > Create a SPARK job > 1)Create a Dataframe using spark.read with the following query > select from > 2)Write the dataframe to a target table with bit type as column. > > Observation : Bit type is getting converted to NULL at the target > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17177465#comment-17177465 ] ZhouDaHong commented on SPARK-32587: 抱歉，我不是特别明白你的问题。你是不是说数据存在空值列的时候，无法写入到sql server数据库？如果是这样的话，请查看待写入的表的结构，查看报错字段是否在数据库中定义了“not null”？ > SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing > NULL values > - > > Key: SPARK-32587 > URL: https://issues.apache.org/jira/browse/SPARK-32587 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 2.4.5 >Reporter: Mohit Dave >Priority: Major > > While writing to a target in SQL Server using Microsoft's SQL Server driver > using dataframe.write API the target is storing NULL values for BIT columns. > > Table definition > Azure SQL DB > 1)Create 2 tables with column type as bit > 2)Insert some record into 1 table > Create a SPARK job > 1)Create a Dataframe using spark.read with the following query > select from > 2)Write the dataframe to a target table with bit type as column. > > Observation : Bit type is getting converted to NULL at the target > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32587) SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing NULL values
[ https://issues.apache.org/jira/browse/SPARK-32587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17176996#comment-17176996 ] Hyukjin Kwon commented on SPARK-32587: -- Could you share the reproducible codes and the actual output? > SPARK SQL writing to JDBC target with bit datatype using Dataframe is writing > NULL values > - > > Key: SPARK-32587 > URL: https://issues.apache.org/jira/browse/SPARK-32587 > Project: Spark > Issue Type: Bug > Components: Spark Core >Affects Versions: 2.4.5 >Reporter: Mohit Dave >Priority: Major > > While writing to a target in SQL Server using Microsoft's SQL Server driver > using dataframe.write API the target is storing NULL values for BIT columns. > > Table definition > Azure SQL DB > 1)Create 2 tables with column type as bit > 2)Insert some record into 1 table > Create a SPARK job > 1)Create a Dataframe using spark.read with the following query > select from > 2)Write the dataframe to a target table with bit type as column. > > Observation : Bit type is getting converted to NULL at the target > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
