[
https://issues.apache.org/jira/browse/HIVE-24822?focusedWorklogId=557915&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-557915
]
ASF GitHub Bot logged work on HIVE-24822:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 25/Feb/21 13:52
Start Date: 25/Feb/21 13:52
Worklog Time Spent: 10m
Work Description: kasakrisz opened a new pull request #2022:
URL: https://github.com/apache/hive/pull/2022
### What changes were proposed in this pull request?
Set `materializationTime` when updating `CreationMetadata` using
`ObjectStore.updateCreationMetadata`
### Why are the changes needed?
When Rebuilding a MaterializedView a new `CreationMetadata` instance is
created for the view which contains updated information like `ValidTxnList`.
Initially this object has a `materializationTime` value with `0` and this is
going to be stored in the `HiveMaterializedViewsRegistry` wrapped by the
`RelOptMaterialization` object represents the view. However the
`materializationTime` value stored in the MetaStore is recalculated and stored
during the update hence MS and the registry has different values. This can lead
unnecessary registry refresh or even `NullPointerException` as the jira
explains.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
```
mvn test
-Dtest=TestObjectStore#testUpdateCreationMetadataSetsMaterializationTime -pl
standalone-metastore/metastore-server
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 557915)
Remaining Estimate: 0h
Time Spent: 10m
> Materialized View rebuild loses materializationTime property value
> ------------------------------------------------------------------
>
> Key: HIVE-24822
> URL: https://issues.apache.org/jira/browse/HIVE-24822
> Project: Hive
> Issue Type: Bug
> Reporter: Krisztian Kasa
> Assignee: Krisztian Kasa
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Materialized View rebuild like
> {code}
> alter materialized view mat1 rebuild;
> {code}
> updates the CreationMetadata of a org.apache.hadoop.hive.ql.metadata.Table
> object of the materialized view but it does not copy the materializationTime
> property value from the original CreationMetadata object and updates the
> entry in the MaterializedViewCache:
> {code}
> } else if (desc.isUpdateCreationMetadata()) {
> // We need to update the status of the creation signature
> Table mvTable = context.getDb().getTable(desc.getName());
> CreationMetadata cm = new
> CreationMetadata(MetaStoreUtils.getDefaultCatalog(context.getConf()),
> mvTable.getDbName(), mvTable.getTableName(),
>
> ImmutableSet.copyOf(mvTable.getCreationMetadata().getTablesUsed()));
>
> cm.setValidTxnList(context.getConf().get(ValidTxnWriteIdList.VALID_TABLES_WRITEIDS_KEY));
> context.getDb().updateCreationMetadata(mvTable.getDbName(),
> mvTable.getTableName(), cm);
> mvTable.setCreationMetadata(cm);
>
> HiveMaterializedViewsRegistry.get().createMaterializedView(context.getDb().getConf(),
> mvTable);
> }
> {code}
> Later when loading Materializetions using
> {code}
> Hive.getValidMaterializedViews(List<Table> materializedViewTables ...)
> {code}
> the materialization stored in the cache and in the metastore will be not the
> same because of the lost materializationTime.
> Cache tried to be refreshed
> {code}
> HiveMaterializedViewsRegistry.get().refreshMaterializedView(conf, null,
> materializedViewTable);
> {code}
> by passing null as oldMaterializedViewTable which leads to
> NullPointerException and CBO failure because the registry expect a non null
> oldMaterializedViewTable value:
> It may drops the old MV in Metastore and also tries to update the cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
