[jira] [Commented] (HIVE-18839) Implement incremental rebuild for materialized views (only insert operations in source tables)

[Ashutosh Chauhan (JIRA)]

    [ 
https://issues.apache.org/jira/browse/HIVE-18839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16430996#comment-16430996
 ] 

Ashutosh Chauhan commented on HIVE-18839:
-----------------------------------------

Failures TestMetaStoreMaterializationsCacheCleaner and 
TestUpdateDeleteSemanticAnalyzer looks related.

> Implement incremental rebuild for materialized views (only insert operations 
> in source tables)
> ----------------------------------------------------------------------------------------------
>
>                 Key: HIVE-18839
>                 URL: https://issues.apache.org/jira/browse/HIVE-18839
>             Project: Hive
>          Issue Type: Improvement
>          Components: Materialized views
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>              Labels: TODOC3.0
>         Attachments: HIVE-18839.01.patch, HIVE-18839.02.patch, 
> HIVE-18839.03.patch, HIVE-18839.patch
>
>
> Implementation will follow current code path for full rebuild. 
> When the MV query plan is retrieved, if the MV contents are outdated because 
> there were insert operations in the source tables, we will introduce a filter 
> with a condition based on stored value of ValidWriteIdLists. For instance, 
> {{WRITE_ID < high_txn_id AND WRITE_ID NOT IN (x, y, ...)}}. Then the 
> rewriting will do the rest of the work by creating a partial rewriting, where 
> the contents of the MV are read as well as the new contents from the source 
> tables.
> This mechanism will not work only for ALTER MV... REBUILD, but also for user 
> queries which will be able to benefit from using outdated MVs to compute part 
> of the needed results.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)