[
https://issues.apache.org/jira/browse/HIVE-27355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17737502#comment-17737502
]
Denys Kuzmenko commented on HIVE-27355:
---------------------------------------
Merged to master.
[~difin], thanks for the patch, and [~dengzh], [~wechar], [~ayushsaxena] for
the review!
> Iceberg: Create table can be slow due to file listing for stats
> ---------------------------------------------------------------
>
> Key: HIVE-27355
> URL: https://issues.apache.org/jira/browse/HIVE-27355
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration
> Reporter: Rajesh Balamohan
> Assignee: Dmitriy Fingerman
> Priority: Major
> Labels: pull-request-available
>
> Stacktrace can be different for hive master branch. But issue is, stats need
> not be populated for iceberg tables and currently it is doing recursive calls
> causing delays during table creation (e.g CTAS).
>
> {noformat}
> at
> org.apache.hadoop.hive.common.FileUtils.listStatusRecursively(FileUtils.java:329)
> at
> org.apache.hadoop.hive.common.FileUtils.listStatusRecursively(FileUtils.java:330)
> at
> org.apache.hadoop.hive.common.FileUtils.listStatusRecursively(FileUtils.java:330)
> at
> org.apache.hadoop.hive.common.HiveStatsUtils.getFileStatusRecurse(HiveStatsUtils.java:61)
> at
> org.apache.hadoop.hive.metastore.Warehouse.getFileStatusesForUnpartitionedTable(Warehouse.java:581)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updateTableStatsFast(MetaStoreUtils.java:201)
> at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.updateTableStatsFast(MetaStoreUtils.java:194)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1445)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1502)
> at sun.reflect.GeneratedMethodAccessor118.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
> at
> org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
> at com.sun.proxy.$Proxy49.create_table_with_environment_context(Unknown
> Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.create_table_with_environment_context(HiveMetaStoreClient.java:2419)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:755)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:743)
> at sun.reflect.GeneratedMethodAccessor117.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
