Jesus Camacho Rodriguez commented on HIVE-18839:

[~ashutoshc], addressed comments and uploaded a new patch to RB and the issue. 
I left a couple of questions in RB.
In turn, after moving one of the old rewriting tests to MiniLlapLocalDriver, 
rewriting is not being triggered for one query (last query in 
{{materialized_view_rewrite_4.q}}), which is weird because logical plan should 
be the same (I am thinking on a different configuration property value that is 
altering the plan, maybe). Issue does not seem related in any case, so I will 
trigger that in a follow-up if that is OK.

> Implement incremental rebuild for materialized views (only insert operations 
> in source tables)
> ----------------------------------------------------------------------------------------------
>                 Key: HIVE-18839
>                 URL: https://issues.apache.org/jira/browse/HIVE-18839
>             Project: Hive
>          Issue Type: Improvement
>          Components: Materialized views
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>              Labels: TODOC3.0
>         Attachments: HIVE-18839.01.patch, HIVE-18839.02.patch, 
> HIVE-18839.patch
> Implementation will follow current code path for full rebuild. 
> When the MV query plan is retrieved, if the MV contents are outdated because 
> there were insert operations in the source tables, we will introduce a filter 
> with a condition based on stored value of ValidWriteIdLists. For instance, 
> {{WRITE_ID < high_txn_id AND WRITE_ID NOT IN (x, y, ...)}}. Then the 
> rewriting will do the rest of the work by creating a partial rewriting, where 
> the contents of the MV are read as well as the new contents from the source 
> tables.
> This mechanism will not work only for ALTER MV... REBUILD, but also for user 
> queries which will be able to benefit from using outdated MVs to compute part 
> of the needed results.

This message was sent by Atlassian JIRA

Reply via email to