Dmitriy Fingerman created HIVE-27883:
----------------------------------------
Summary: Hive Iceberg insert query doesn't merge small files
Key: HIVE-27883
URL: https://issues.apache.org/jira/browse/HIVE-27883
Project: Hive
Issue Type: Bug
Components: Hive, Iceberg integration
Reporter: Dmitriy Fingerman
Attachments: iceberg_insert_no_compaction.q,
iceberg_insert_no_compaction.q.out.orig
The attached hive query test file reproduces insertion of multiple small data
files into Iceberg table instead of one combined data file
The attached output file shows the data files, before and after.
Setting table property *'write.target-file-size-bytes'* doesn't make a
difference as well as other Hive settings like below that work for Hive native
tables.
{code:java}
set hive.merge.tezfiles=true;
set hive.merge.mapfiles=true;
set hive.merge.mapredfiles=true;
set hive.merge.size.per.task=128000000;
set hive.merge.smallfiles.avgsize=128000000;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
