[
https://issues.apache.org/jira/browse/HIVE-27883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Fingerman updated HIVE-27883:
-------------------------------------
Attachment: iceberg_insert_no_compaction.q
iceberg_insert_no_compaction.q.out.orig
> Hive Iceberg insert query doesn't merge small files
> ---------------------------------------------------
>
> Key: HIVE-27883
> URL: https://issues.apache.org/jira/browse/HIVE-27883
> Project: Hive
> Issue Type: Bug
> Components: Hive, Iceberg integration
> Reporter: Dmitriy Fingerman
> Priority: Major
> Attachments: iceberg_insert_no_compaction.q,
> iceberg_insert_no_compaction.q.out.orig
>
>
> The attached hive query test file reproduces insertion of multiple small data
> files into Iceberg table instead of one combined data file
> The attached output file shows the data files, before and after.
> Setting table property *'write.target-file-size-bytes'* doesn't make a
> difference as well as other Hive settings like below that work for Hive
> native tables.
> {code:java}
> set hive.merge.tezfiles=true;
> set hive.merge.mapfiles=true;
> set hive.merge.mapredfiles=true;
> set hive.merge.size.per.task=128000000;
> set hive.merge.smallfiles.avgsize=128000000;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
