[
https://issues.apache.org/jira/browse/IMPALA-11378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17573970#comment-17573970
]
ASF subversion and git services commented on IMPALA-11378:
----------------------------------------------------------
Commit c0b0875bda59771fb1b5c55a5eaf45f3dcfaa63c in impala's branch
refs/heads/master from Tamas Mate
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=c0b0875bd ]
IMPALA-11378: Allow INSERT OVERWRITE for bucket tranforms in some cases
This change has been considered only for Iceberg tables mainly for table
maintenance reasons. Iceberg table writes create new snapshots and these
can accumulate over time. This commit allows a simple form of compaction
of these snapshots.
INSERT OVERWRITES have been blocked in case partition evolution is in
place, because it would be possible to overwrite a data file with a
newer schema that has less columns. This could cause unexpected data
loss.
For bucketed tables, the following syntax is allowed to be executed:
INSERT OVERWRITE ice_tbl SELECT * FROM ice_tbl;
The source and target table has to be the same and specified, only
SELECT '*' queries are allowed. These requirements are also in place to
avoid unexpected data loss.
- Values are not allowed, because inserting a single record could
overwrite a whole file in a bucket.
- Only source table is allowed, because at the time of the insert it
is unknown which files will be modified, similar to values.
Testing:
- Added e2e tests.
Change-Id: Ibd1bc19d839297246eadeb754cdeeec1e306098a
Reviewed-on: http://gerrit.cloudera.org:8080/18649
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Allow INSERT OVERWRITE into bucket partition transform in some cases
> --------------------------------------------------------------------
>
> Key: IMPALA-11378
> URL: https://issues.apache.org/jira/browse/IMPALA-11378
> Project: IMPALA
> Issue Type: Sub-task
> Components: Frontend
> Affects Versions: Impala 4.1.0
> Reporter: Tamas Mate
> Assignee: Tamas Mate
> Priority: Major
> Labels: impala-iceberg
>
> As said in the parent Jira, in some cases bucket partitioning can be allowed
> safely:
> * there were no partition evolutions
> * the source and destination table is the same
> * overwriting with static values
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
