[ 
https://issues.apache.org/jira/browse/SPARK-28495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wenchen Fan reassigned SPARK-28495:
-----------------------------------

    Assignee: Gengliang Wang

> Introduce ANSI store assignment policy for table insertion
> ----------------------------------------------------------
>
>                 Key: SPARK-28495
>                 URL: https://issues.apache.org/jira/browse/SPARK-28495
>             Project: Spark
>          Issue Type: Sub-task
>          Components: SQL
>    Affects Versions: 3.0.0
>            Reporter: Gengliang Wang
>            Assignee: Gengliang Wang
>            Priority: Major
>
> In Spark version 2.4 and earlier, when inserting into a table, Spark will 
> cast the data type of input query to the data type of target table by 
> coercion. This can be super confusing, e.g. users make a mistake and write 
> string values to an int column.
> In data source V2,  by default, only upcasting is allowed when inserting data 
> into a table. E.g. int -> long and int -> string are allowed, while decimal 
> -> double or long -> int are not allowed. The rules of UpCast was originally 
> created for Dataset type coercion. They are quite strict and different from 
> the behavior of all existing popular DBMS. This is breaking change. It is 
> possible that existing queries are broken after 3.0 releases.
> Following ANSI SQL standard makes Spark consistent with the table insertion 
> behaviors of popular DBMS like PostgreSQL/Oracle/Mysql.
> For more details, see the discussion on 
> http://apache-spark-developers-list.1001551.n3.nabble.com/Discuss-Follow-ANSI-SQL-on-table-insertion-td27531.html#a27562
>  and https://github.com/apache/spark/pull/25453 .
> This task is to add ANSI store assignment policy as a new option for the 
> configuration "spark.sql.storeAssignmentPolicy“



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to