[
https://issues.apache.org/jira/browse/HIVE-28941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-28941:
----------------------------------
Summary: Do not create empty commit files in HiveIcebergOutputCommitter
(was: Do not create empty commit files)
> Do not create empty commit files in HiveIcebergOutputCommitter
> --------------------------------------------------------------
>
> Key: HIVE-28941
> URL: https://issues.apache.org/jira/browse/HIVE-28941
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
> Reporter: Denys Kuzmenko
> Priority: Major
>
> {code}
> 2025-04-23T14:53:02.704Z query-executor-0-15 query-executor 1
> e73419bb-15eb-4346-8009-f160282ee340 [mdc@38374
> class="hive.HiveIcebergOutputCommitter" dagId="dag_1745051473706_0016_579"
> fragmentId="1745051473706_0016_579_05_000015_0" level="INFO"
> queryId="hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd"
> thread="TezTR-473706_16_579_5_15_0"] Created Iceberg commitTask manifest
> file:
> s3a://s3-cdp-dub-prd-int-prd-464373096835/data/warehouse/tablespace/external/hive/adp_staging.db/fct_account_cashflow_forecast/temp/hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd-job_17450514737065_0016/task-15.forCommit\rFilesForCommit\{dataFiles=[],
> deleteFiles=[], replacedDataFiles=[], referencedDataFiles=[]}
> 2025-04-23T14:53:02.751Z query-executor-0-15 query-executor 1
> e73419bb-15eb-4346-8009-f160282ee340 [mdc@38374
> class="hive.HiveIcebergOutputCommitter" dagId="dag_1745051473706_0016_579"
> fragmentId="1745051473706_0016_579_05_000005_0" level="INFO"
> queryId="hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd"
> thread="TezTR-473706_16_579_5_5_0"] Created Iceberg commitTask manifest file:
> s3a://s3-cdp-dub-prd-int-prd-464373096835/data/warehouse/tablespace/external/hive/adp_staging.db/fct_account_cashflow_forecast/temp/hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd-job_17450514737065_0016/task-5.forCommit\rFilesForCommit\{dataFiles=[],
> deleteFiles=[], replacedDataFiles=[], referencedDataFiles=[]}
> 2025-04-23T14:53:04.778Z query-executor-0-15 query-executor 1
> e73419bb-15eb-4346-8009-f160282ee340 [mdc@38374
> class="hive.HiveIcebergOutputCommitter" dagId="dag_1745051473706_0016_579"
> fragmentId="1745051473706_0016_579_05_000015_2" level="INFO"
> queryId="hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd"
> thread="TezTR-473706_16_579_5_15_2"] Created Iceberg commitTask manifest
> file:
> s3a://s3-cdp-dub-prd-int-prd-464373096835/data/warehouse/tablespace/external/hive/adp_staging.db/fct_account_cashflow_forecast/temp/hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd-job_17450514737065_0016/task-15.forCommit\rFilesForCommit\{dataFiles=[],
> deleteFiles=[], replacedDataFiles=[], referencedDataFiles=[]}
> 2025-04-23T14:53:04.826Z query-executor-0-15 query-executor 1
> e73419bb-15eb-4346-8009-f160282ee340 [mdc@38374
> class="hive.HiveIcebergOutputCommitter" dagId="dag_1745051473706_0016_579"
> fragmentId="1745051473706_0016_579_05_000003_20" level="INFO"
> queryId="hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd"
> thread="TezTR-473706_16_579_5_3_20"] Created Iceberg commitTask manifest
> file:
> s3a://s3-cdp-dub-prd-int-prd-464373096835/data/warehouse/tablespace/external/hive/adp_staging.db/fct_account_cashflow_forecast/temp/hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd-job_17450514737065_0016/task-3.forCommit\rFilesForCommit\{dataFiles=[],
> deleteFiles=[], replacedDataFiles=[], referencedDataFiles=[]}
> 2025-04-23T14:53:04.876Z query-executor-0-15 query-executor 1
> e73419bb-15eb-4346-8009-f160282ee340 [mdc@38374
> class="hive.HiveIcebergOutputCommitter" dagId="dag_1745051473706_0016_579"
> fragmentId="1745051473706_0016_579_05_000005_2" level="INFO"
> queryId="hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd"
> thread="TezTR-473706_16_579_5_5_2"] Created Iceberg commitTask manifest file:
> s3a://s3-cdp-dub-prd-int-prd-464373096835/data/warehouse/tablespace/external/hive/adp_staging.db/fct_account_cashflow_forecast/temp/hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd-job_17450514737065_0016/task-5.forCommit\rFilesForCommit\{dataFiles=[],
> deleteFiles=[], replacedDataFiles=[], referencedDataFiles=[]}
> 2025-04-23T14:53:04.909Z query-executor-0-15 query-executor 1
> e73419bb-15eb-4346-8009-f160282ee340 [mdc@38374
> class="hive.HiveIcebergOutputCommitter" dagId="dag_1745051473706_0016_579"
> fragmentId="1745051473706_0016_579_05_000011_2" level="INFO"
> queryId="hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd"
> thread="TezTR-473706_16_579_5_11_2"] Created Iceberg commitTask manifest
> file:
> s3a://s3-cdp-dub-prd-int-prd-464373096835/data/warehouse/tablespace/external/hive/adp_staging.db/fct_account_cashflow_forecast/temp/hive_20250423145215_4f1655f3-a739-4d0c-8edb-c2ab4fda6cfd-job_17450514737065_0016/task-11.forCommit\rFilesForCommit\{dataFiles=[],
> deleteFiles=[], replacedDataFiles=[], referencedDataFiles=[]}
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
