[
https://issues.apache.org/jira/browse/HIVE-27101?focusedWorklogId=849571&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-849571
]
ASF GitHub Bot logged work on HIVE-27101:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 07/Mar/23 14:34
Start Date: 07/Mar/23 14:34
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4079:
URL: https://github.com/apache/hive/pull/4079#discussion_r1127959641
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -2101,6 +2102,41 @@ public boolean
validateMaterializedViewsFromRegistry(List<Table> cachedMateriali
}
}
+ private Materialization
getMaterializationInvalidationInfo(MaterializedViewMetadata metadata)
+ throws TException, HiveException {
+ MaterializationSnapshot mvSnapshot =
MaterializationSnapshot.fromJson(metadata.creationMetadata.getValidTxnList());
+ if (mvSnapshot.getTableSnapshots() == null ||
mvSnapshot.getTableSnapshots().isEmpty()) {
+ return getMSC().getMaterializationInvalidationInfo(
+ metadata.creationMetadata, conf.get(ValidTxnList.VALID_TXNS_KEY));
+ }
+
+ boolean hasDelete = false;
+ for (SourceTable sourceTable : metadata.getSourceTables()) {
+ Table table = getTable(sourceTable.getTable().getDbName(),
sourceTable.getTable().getTableName());
+ HiveStorageHandler storageHandler = table.getStorageHandler();
+ if (storageHandler == null) {
+ Materialization materialization = new Materialization();
+ materialization.setSourceTablesCompacted(true);
+ return materialization;
+ }
+ Boolean b = storageHandler.hasDeleteOperation(
+ table,
mvSnapshot.getTableSnapshots().get(table.getFullyQualifiedName()));
+ if (b == null) {
+ Materialization materialization = new Materialization();
+ materialization.setSourceTablesCompacted(true);
+ return materialization;
Review Comment:
In this case `null` means we can not confirm that this source table has
deletes or not. This means the incremental rebuild plan can not be chosen so
fall back to full rebuild.
If it can not be confirmed at least one of the source tables we fall back to
full rebuild.
Issue Time Tracking
-------------------
Worklog Id: (was: 849571)
Time Spent: 5h 50m (was: 5h 40m)
> Support incremental materialized view rebuild when Iceberg source tables have
> insert operation only.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-27101
> URL: https://issues.apache.org/jira/browse/HIVE-27101
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
