Tamas Mate has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/18649 )
Change subject: IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases ...................................................................... IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases This change has been considered only for Iceberg tables mainly for table maintenance reasons. Iceberg table writes create new snapshots and these can accumulate over time. This commit allows a simple form of compaction of these snapshots. INSERT OVERWRITES have been blocked in case partition evolution is in place, because it would be possible to overwrite a data file with a newer schema that has less columns. This could cause unexpected data loss. For bucketed tables, the following syntax is allowed to be executed: INSERT OVERWRITE ice_tbl SELECT * FROM ice_tbl; The source and target table has to be the same and specified, only SELECT '*' queries are allowed. These requirements are also in place to avoid unexpected data loss. - Values are not allowed, because inserting a single record could overwrite a whole file in a bucket. - Only source table is allowed, because at the time of the insert it is unknown which files will be modified, similar to values. Testing: - Added e2e tests. Change-Id: Ibd1bc19d839297246eadeb754cdeeec1e306098a --- M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M testdata/workloads/functional-query/queries/QueryTest/iceberg-negative.test M testdata/workloads/functional-query/queries/QueryTest/iceberg-overwrite.test M tests/custom_cluster/test_events_custom_configs.py 4 files changed, 112 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/49/18649/7 -- To view, visit http://gerrit.cloudera.org:8080/18649 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ibd1bc19d839297246eadeb754cdeeec1e306098a Gerrit-Change-Number: 18649 Gerrit-PatchSet: 7 Gerrit-Owner: Tamas Mate <[email protected]> Gerrit-Reviewer: Gabor Kaszab <[email protected]> Gerrit-Reviewer: Gergely Fürnstáhl <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Tamas Mate <[email protected]> Gerrit-Reviewer: Zoltan Borok-Nagy <[email protected]>
