kasakrisz commented on code in PR #4079:
URL: https://github.com/apache/hive/pull/4079#discussion_r1133589995
##########
ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java:
##########
@@ -2101,6 +2102,55 @@ public boolean
validateMaterializedViewsFromRegistry(List<Table> cachedMateriali
}
}
+ private Materialization
getMaterializationInvalidationInfo(MaterializedViewMetadata metadata)
+ throws TException, HiveException {
+ Optional<SourceTable> first =
metadata.getSourceTables().stream().findFirst();
+ if (!first.isPresent()) {
+ // This is unexpected: all MV must have at least one source
+ Materialization materialization = new Materialization();
+ materialization.setSourceTablesCompacted(true);
+ materialization.setSourceTablesUpdateDeleteModified(true);
+ return new Materialization();
+ } else {
+ Table table = getTable(first.get().getTable().getDbName(),
first.get().getTable().getTableName());
+ if (!(table.isNonNative() &&
table.getStorageHandler().areSnapshotsSupported())) {
+ // Mixing native and non-native acid source tables are not supported.
If the first source is native acid
+ // the rest is expected to be native acid
+ return getMSC().getMaterializationInvalidationInfo(
+ metadata.creationMetadata,
conf.get(ValidTxnList.VALID_TXNS_KEY));
+ }
+ }
+
+ MaterializationSnapshot mvSnapshot =
MaterializationSnapshot.fromJson(metadata.creationMetadata.getValidTxnList());
+
+ boolean hasDelete = false;
+ for (SourceTable sourceTable : metadata.getSourceTables()) {
+ Table table = getTable(sourceTable.getTable().getDbName(),
sourceTable.getTable().getTableName());
+ HiveStorageHandler storageHandler = table.getStorageHandler();
+ if (storageHandler == null) {
+ Materialization materialization = new Materialization();
+ materialization.setSourceTablesCompacted(true);
+ return materialization;
+ }
+ Boolean b = storageHandler.hasDeletes(
Review Comment:
I changed the api method name to hasAppendsOnly and the semantics since it
is more accurate here. This patch supports incremental mv rebuild only if
source tables has insert (append) operations only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
