[
https://issues.apache.org/jira/browse/HUDI-5317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Qijun Fu updated HUDI-5317:
---------------------------
Description:
For sql like insert overwrite table $table select xxx, we expect to drop all
data in the table first and then insert the selected data into it. But we
found that the 'insert overwrite table' semantic works only for non-partitioned
table. For partitioned table, current implementation will drop only partitions
involved in the select sub-query, other partitions won't be dropped( which
should be dropped as expected).
We will submit a pr to solve the problem that insert overwrite table can drop
all partitions at first then insert new data.
was:
For sql like insert overwrite table $table select xxx, we expect to drop all
data in the table first and then insert the selected data into it. But we
found that the 'insert overwrite table' semantic works only for non-partitioned
table. For partitioned table, current implementation will drop only partitions
involved in the select sub-query, other partitions won't be dropped( which
should be dropped as expected).
We will submit a pr to solve the problem.
> SparkSQL insert overwrite table doesn't work as expected for partitioned table
> ------------------------------------------------------------------------------
>
> Key: HUDI-5317
> URL: https://issues.apache.org/jira/browse/HUDI-5317
> Project: Apache Hudi
> Issue Type: Bug
> Components: spark-sql
> Reporter: Qijun Fu
> Assignee: Qijun Fu
> Priority: Minor
>
> For sql like insert overwrite table $table select xxx, we expect to drop all
> data in the table first and then insert the selected data into it. But we
> found that the 'insert overwrite table' semantic works only for
> non-partitioned table. For partitioned table, current implementation will
> drop only partitions involved in the select sub-query, other partitions won't
> be dropped( which should be dropped as expected).
> We will submit a pr to solve the problem that insert overwrite table can
> drop all partitions at first then insert new data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
