Jesus Camacho Rodriguez commented on HIVE-18825:

I have been checking the code, and indeed changing the position for acquiring 
the locks requires a lot of work. Although we extract the read entities while 
parsing (and I guess we could extend that logic and extract write entities 
too), the logic to extract locks, etc., is tightly linked to the query plan 

One question I have is: does the proposed change break any current existing use 
case or do we need it only in case we want to support lock based concurrency 
If it does not break the current system implementation, I suggest we commit 
this fix to unlock MV incremental rebuild, etc., and we create a follow-up JIRA 
to move the logic to acquire locks to be executed after parsing and before rest 
of compilation.

> Define ValidTxnList before starting query optimization
> ------------------------------------------------------
>                 Key: HIVE-18825
>                 URL: https://issues.apache.org/jira/browse/HIVE-18825
>             Project: Hive
>          Issue Type: Improvement
>          Components: Transactions
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>            Priority: Major
>         Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch, 
> HIVE-18825.03.patch, HIVE-18825.04.patch, HIVE-18825.patch
> Consider a set of tables used by a materialized view where inserts happened 
> after the materialization was created. To compute incremental view 
> maintenance, we need to be able to filter only new rows from those base 
> tables. That can be done by inserting a filter operator with condition e.g. 
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT 
> IN(<open txns>)}} on top of the MVs query definition and triggering the 
> rewriting (which should in turn produce a partial rewriting). However, to do 
> that, we need to have a value for {{ValidTxnList}} during query compilation 
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There 
> should not be any visible changes for end user.

This message was sent by Atlassian JIRA

Reply via email to