[
https://issues.apache.org/jira/browse/HIVE-27185?focusedWorklogId=853891&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-853891
]
ASF GitHub Bot logged work on HIVE-27185:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Mar/23 10:21
Start Date: 30/Mar/23 10:21
Worklog Time Spent: 10m
Work Description: ayushtkn commented on code in PR #4165:
URL: https://github.com/apache/hive/pull/4165#discussion_r1153050938
##########
iceberg/iceberg-handler/src/main/java/org/apache/iceberg/mr/hive/IcebergTableUtil.java:
##########
@@ -52,12 +52,13 @@ private IcebergTableUtil() {
* Constructs the table properties needed for the Iceberg table loading by
retrieving the information from the
* hmsTable. It then calls {@link IcebergTableUtil#getTable(Configuration,
Properties)} with these properties.
* @param configuration a Hadoop configuration
- * @param hmsTable the HMS table
- * @param skipCache if set to true there won't be an attempt to retrieve the
table from SessionState
+ * @param hmsTable the HMS table
+ * @param skipCache if set to true there won't be an attempt to retrieve
the table from SessionState
+ * @param suffix the suffix to use for cache.
* @return the Iceberg table
*/
static Table getTable(Configuration configuration,
org.apache.hadoop.hive.metastore.api.Table hmsTable,
- boolean skipCache) {
+ boolean skipCache, String suffix) {
Review Comment:
We want to cache the table for Stats only, if you fetch once for Stats, then
use that till end, if you fetch for something else, we don't use that.
In most cases that won't be a problem, but the way we do CTAS and MV's it
did creates problem. It first creates an iceberg table and then does the load
into that, so in that cases caching would have stale table. Got a test failure
as well for those MV cases in the last run
Issue Time Tracking
-------------------
Worklog Id: (was: 853891)
Time Spent: 50m (was: 40m)
> Iceberg: Cache iceberg table while loading for stats
> ----------------------------------------------------
>
> Key: HIVE-27185
> URL: https://issues.apache.org/jira/browse/HIVE-27185
> Project: Hive
> Issue Type: Improvement
> Reporter: Ayush Saxena
> Assignee: Ayush Saxena
> Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Presently iceberg for stats loads the iceberg table multiple times for stats
> via different routes.
> Cache it to avoid reading/loading the iceberg table multiple times.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
