[
https://issues.apache.org/jira/browse/SPARK-50668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kevinshin updated SPARK-50668:
------------------------------
Summary: Got UNSUPPORTED_INSERT.RDD_BASED error when insert into a jdbc
datasource (was: UNSUPPORTED_INSERT.RDD_BASED when insert into a jdbc
datasource)
> Got UNSUPPORTED_INSERT.RDD_BASED error when insert into a jdbc datasource
> -------------------------------------------------------------------------
>
> Key: SPARK-50668
> URL: https://issues.apache.org/jira/browse/SPARK-50668
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.5.4
> Environment: hdp 3.1.4
> spark 3.5.4
> Reporter: kevinshin
> Priority: Major
>
> I have some SQL scripts for pushing the calculation results of the data
> warehouse to the JDBC DATASOURCE.
> When I attempted to upgrade from Spark 3.3.3 to Spark 3.5.4, I found that one
> of them failed to run properly: it pushed data to the RDS database of Alibaba
> Cloud, but other similar scripts worked without any issues.
> This script can run normally on both Spark 3.3.3 and 3.4.4.
>
> SCRIPT:
> {code:java}
> CREATE TEMPORARY VIEW sg_apdata_v_prd
> USING org.apache.spark.sql.jdbc
> OPTIONS (
> url
> "jdbc:mysql://rm-abc.mysql.rds.aliyuncs.com:3306/db_01?useUnicode=true&characterEncoding=UTF-8&useSSL=false",
> dbtable "`apdata`",
> user 'sgroot',
> password '####'
> );{code}
>
> {code:java}
> desc sg_apdata_v_prd;
> user_name string
> region string
> ap_name string
> dtime string
> ddate string{code}
>
> {code:java}
> explain
> INSERT INTO sg_apdata_v_prd
> select 'name' as user_name, 'region' as region, 'ap_name' as ap_name,
> '2024-12-24 23:45:00' as dtime, '2024-12-24' as ddate;{code}
>
> {code:java}
> == Physical Plan ==
>
>
> org.apache.spark.sql.AnalysisException: [UNSUPPORTED_INSERT.RDD_BASED] Can't
> insert into the target. An RDD-based table is not allowed.;
>
> 'InsertIntoStatement Project [user_name#0, region#1, ap_name#2, dtime#3,
> staticinvoke(class
> org.apache.spark.sql.catalyst.util.CharVarcharCodegenUtils, StringType,
> readSidePadding, ddate#4, 10, true, false, true) AS ddate#5], false, false,
> false
>
> +-
> Project [name AS user_name#67, region AS region#68, ap_name AS ap_name#69,
> 2024-12-24 23:45:00 AS dtime#70, 2024-12-24 AS ddate#71]
>
> +- OneRowRelation {code}
>
> Spark 3.4.4 Physical Plan is :
> {code:java}
> == Physical Plan == Execute InsertIntoDataSourceCommand +-
> InsertIntoDataSourceCommand Relation
> [user_name#0,region#1,ap_name#2,dtime#3,ddate#4] JDBCRelation(`sg-apdata`)
> [numPartitions=1], false +- Project [cast(user_name#10 as string) AS
> user_name#21, cast(region#11 as string) AS region#22, cast(ap_name#12 as
> string) AS ap_name#23, cast(dtime#13 as string) AS dtime#24, cast(ddate#14 as
> string) AS ddate#25] +- Project [name AS user_name#10, region AS region#11,
> ap_name AS ap_name#12, 2024-12-24 23:45:00 AS dtime#13, 2024-12-24 AS
> ddate#14] +- OneRowRelation{code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
