-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/72528/
-----------------------------------------------------------
(Updated June 9, 2020, 8:52 a.m.)
Review request for hive, Jesús Camacho Rodríguez, Peter Varga, and Peter Vary.
Bugs: HIVE-23503
https://issues.apache.org/jira/browse/HIVE-23503
Repository: hive-git
Description
-------
ValidTxnManager doesn't consider txns opened and committed between snapshot
generation and locking when evaluating ValidTxnListState. This cause issues
like duplicate insert in case of concurrent merge insert & insert.
Diffs
-----
ql/src/java/org/apache/hadoop/hive/ql/Driver.java e70c92eef4
ql/src/java/org/apache/hadoop/hive/ql/DriverContext.java a8c83fc504
ql/src/java/org/apache/hadoop/hive/ql/ValidTxnManager.java 7d49c57dda
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbTxnManager.java 71afcbdc68
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DummyTxnManager.java 0383881acc
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/HiveTxnManager.java 600289f837
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java
8a15b7cc5d
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
65df9c2ba9
standalone-metastore/metastore-common/src/main/java/org/apache/hadoop/hive/metastore/IMetaStoreClient.java
887d4303f4
standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClientPreCatalog.java
312936efa8
storage-api/src/java/org/apache/hadoop/hive/common/ValidReadTxnList.java
b8ff03f9c4
storage-api/src/java/org/apache/hadoop/hive/common/ValidTxnList.java
d4c3b09730
Diff: https://reviews.apache.org/r/72528/diff/1/
Testing (updated)
-------
DbTxnManager tests.
Faulty scenario:
1. open and generate snapshot for t1 that merge inserts data from a source
table into the target one.
2. Open, run and commit t2 that inserts source table data into the target table.
3. Run t1 - duplicate date would be inserted into target table as t2 changes
won't be visible by t1.
Thanks,
Denys Kuzmenko
