[
https://issues.apache.org/jira/browse/HIVE-18825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387145#comment-16387145
]
Eugene Koifman edited comment on HIVE-18825 at 3/6/18 2:02 AM:
---------------------------------------------------------------
I think there is a problem here that I should've thought of earlier.
Right now we lock in the snapshot after lock acquisition. This ordering is
important if we ever want to support lock based concurrency control.
(Something I think we should do)
Suppose you have 2 concurrent transactions both running "update T1 set x = x +
1". If we acquire the update lock first, then record the snapshot, then 2nd
txn to get the lock will see the result of write of the previous one. If we
lock the snapshot before acquiring the lock, both transactions may lock in
exactly the same snapshot and locking becomes useless as the 2nd will still
read an "old" snapshot.
Could the predicates you want be inserted at compile time, but bound to actual
values as some post processing after (or at the end of)
{{Driver.acquireLocks()}} as currently implemented?
was (Author: ekoifman):
I think there is a problem here that I should've thought of earlier.
Right now we lock in the snapshot after lock acquisition. This ordering is
important if we ever want to support lock based concurrency control.
Suppose you have 2 concurrent transactions both running "update T1 set x = x +
1". If we acquire the update lock first, then record the snapshot, then 2nd
txn to get the lock will see the result of write of the previous one. If we
lock the snapshot before acquiring the lock, both transactions may lock in
exactly the same snapshot and locking becomes useless as the 2nd will still
read an "old" snapshot.
Could the predicates you want be inserted at compile time, but bound to actual
values as some post processing after (or at the end of)
{{Driver.acquireLocks()}} as currently implemented?
> Define ValidTxnList before starting query optimization
> ------------------------------------------------------
>
> Key: HIVE-18825
> URL: https://issues.apache.org/jira/browse/HIVE-18825
> Project: Hive
> Issue Type: Improvement
> Components: Transactions
> Affects Versions: 3.0.0
> Reporter: Jesus Camacho Rodriguez
> Assignee: Jesus Camacho Rodriguez
> Priority: Major
> Attachments: HIVE-18825.01.patch, HIVE-18825.02.patch,
> HIVE-18825.03.patch, HIVE-18825.04.patch, HIVE-18825.patch
>
>
> Consider a set of tables used by a materialized view where inserts happened
> after the materialization was created. To compute incremental view
> maintenance, we need to be able to filter only new rows from those base
> tables. That can be done by inserting a filter operator with condition e.g.
> {{ROW\_\_ID.transactionId < highwatermark and ROW\_\_ID.transactionId NOT
> IN(<open txns>)}} on top of the MVs query definition and triggering the
> rewriting (which should in turn produce a partial rewriting). However, to do
> that, we need to have a value for {{ValidTxnList}} during query compilation
> so we know the snapshot that we are querying.
> This patch aims to generate {{ValidTxnList}} before query optimization. There
> should not be any visible changes for end user.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
