[
https://issues.apache.org/jira/browse/HUDI-6828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17762559#comment-17762559
]
Jing Zhang commented on HUDI-6828:
----------------------------------
The root cause is currently HUDI insert does not distinguish dynamic partition
insert and static partition insert.
Currently, if writestatus is empty RDD, partitionToReplaceIds mapping is empty,
so the replace commit does not contain old files of overwritten partitions.
> After Insert overwrite into a non-empty partition/table with empty datasets,
> the subsequent query results are incorrect
> -----------------------------------------------------------------------------------------------------------------------
>
> Key: HUDI-6828
> URL: https://issues.apache.org/jira/browse/HUDI-6828
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Jing Zhang
> Priority: Major
>
> After Insert overwrite into a non-empty partition/table with empty datasets,
> the subsequent query results are incorrect
> For example, current dataset of partition dt=2021-07-17 is
> ||id||name||dt||
> |1|'a'|'2021-07-17'|
>
> Using the following SQL to overwrite partition dt=2021-07-17
> {code:java}
> insert overwrite table targetT partition(dt='2021-07-17') select id, name,
> price from sourceT where dt=2021-07-17 and product='test'
> {code}
> If the input dataset is empty. We expect after insert overwrite, the
> partition of dt=2021-07-17 is empty.
> However, the query result is still the old data.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
