[
https://issues.apache.org/jira/browse/HIVE-27101?focusedWorklogId=850566&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-850566
]
ASF GitHub Bot logged work on HIVE-27101:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Mar/23 08:35
Start Date: 13/Mar/23 08:35
Worklog Time Spent: 10m
Work Description: kasakrisz commented on code in PR #4079:
URL: https://github.com/apache/hive/pull/4079#discussion_r1133589995
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -2101,6 +2102,55 @@ public boolean
validateMaterializedViewsFromRegistry(List<Table> cachedMateriali
}
}
+ private Materialization
getMaterializationInvalidationInfo(MaterializedViewMetadata metadata)
+ throws TException, HiveException {
+ Optional<SourceTable> first =
metadata.getSourceTables().stream().findFirst();
+ if (!first.isPresent()) {
+ // This is unexpected: all MV must have at least one source
+ Materialization materialization = new Materialization();
+ materialization.setSourceTablesCompacted(true);
+ materialization.setSourceTablesUpdateDeleteModified(true);
+ return new Materialization();
+ } else {
+ Table table = getTable(first.get().getTable().getDbName(),
first.get().getTable().getTableName());
+ if (!(table.isNonNative() &&
table.getStorageHandler().areSnapshotsSupported())) {
+ // Mixing native and non-native acid source tables are not supported.
If the first source is native acid
+ // the rest is expected to be native acid
+ return getMSC().getMaterializationInvalidationInfo(
+ metadata.creationMetadata,
conf.get(ValidTxnList.VALID_TXNS_KEY));
+ }
+ }
+
+ MaterializationSnapshot mvSnapshot =
MaterializationSnapshot.fromJson(metadata.creationMetadata.getValidTxnList());
+
+ boolean hasDelete = false;
+ for (SourceTable sourceTable : metadata.getSourceTables()) {
+ Table table = getTable(sourceTable.getTable().getDbName(),
sourceTable.getTable().getTableName());
+ HiveStorageHandler storageHandler = table.getStorageHandler();
+ if (storageHandler == null) {
+ Materialization materialization = new Materialization();
+ materialization.setSourceTablesCompacted(true);
+ return materialization;
+ }
+ Boolean b = storageHandler.hasDeletes(
Review Comment:
I changed the api method name to hasAppendsOnly and the semantics since it
is more accurate here. This patch supports incremental mv rebuild only if
source tables has insert (append) operations only.
Issue Time Tracking
-------------------
Worklog Id: (was: 850566)
Time Spent: 7.5h (was: 7h 20m)
> Support incremental materialized view rebuild when Iceberg source tables have
> insert operation only.
> ----------------------------------------------------------------------------------------------------
>
> Key: HIVE-27101
> URL: https://issues.apache.org/jira/browse/HIVE-27101
> Project: Hive
> Issue Type: Improvement
> Components: Iceberg integration, Materialized views
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Labels: pull-request-available
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
