Peter Vary created HIVE-22116:
---------------------------------
Summary: MaterializedView refresh check might return incorrect
result when Compaction is run
Key: HIVE-22116
URL: https://issues.apache.org/jira/browse/HIVE-22116
Project: Hive
Issue Type: Bug
Components: Materialized views
Reporter: Peter Vary
Reading the code of TxnHandler.getMaterializationInvalidationInfo I see that we
decide on the freshness of the view based on the COMPLETED_TXN_COMPONENTS
table.
See:
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java#L2021]
On the other hand if we run a major compaction we clean up
COMPLETED_TXN_COMPONENTS table, so we lose all previous information. We do it
in CompactionTxnHandler.markCleaned.
See:
[https://github.com/apache/hive/blob/master/standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/CompactionTxnHandler.java#L382]
When the following sequence of events happen we do not refresh the materialized
view:
- Create Table
- Create MV
- Refresh MV
- Update Table
- Start major compaction
- Wait until compacted, and cleaned
- Select Table
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
