Re: Review Request 72387: Locks: Add new lock implementations for always zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72387/ --- (Updated April 21, 2020, 12:14 p.m.) Review request for hive, Marton Bod and

Re: Review Request 72388: HIVE-23048 Use sequences for TXN_ID generation

> On April 20, 2020, 10:40 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Lines 902 (patched) > > > > > > Can it be null?

Re: Review Request 72378: HIVE-23201: Improve logging in locking

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72378/#review220389 --- LGTM, just number of comments

Re: Review Request 72378: HIVE-23201: Improve logging in locking

> On April 21, 2020, 10:38 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Lines 4427 (patched) > > > > > > I would keep

Review Request 72387: Locks: Add new lock implementations for always zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72387/ --- Review request for hive, Marton Bod and Peter Vary. Bugs: HIVE-19369

Re: Review Request 72388: HIVE-23048 Use sequences for TXN_ID generation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72388/#review220367 --- LGTM, just a few comments

Re: Review Request 72388: HIVE-23048 Use sequences for TXN_ID generation

> On April 20, 2020, 12:19 p.m., Peter Vary wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Lines 780 (patched) > > > > > > TXN_META_INFO? What

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72112/#review220213 ---

Re: Review Request 72264: HIVE-23052: Optimize lock enqueueing in TxnHandler

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72264/#review220139 --- LGTM, 1 small comment, for your consideration, otherwise - ship

Re: Review Request 72290: HIVE-23067: Use batch DB calls in TxnHandler for commitTxn and abortTxns

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72290/#review220164 --- LGTM, just a few comments.

Re: Review Request 72283: HIVE-23076 Add batching for openTxn

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72283/#review220103 --- LGTM, just minor comments

Re: Review Request 72283: HIVE-23076 Add batching for openTxn

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72283/#review220185 --- Ship it! Ship It! - Denys Kuzmenko On April 2, 2020, 10:23

Re: Review Request 72290: HIVE-23067: Use batch DB calls in TxnHandler for commitTxn and abortTxns

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72290/#review220186 --- Ship it! Ship It! - Denys Kuzmenko On April 1, 2020, 6:53

Re: Review Request 72264: HIVE-23052: Optimize lock enqueueing in TxnHandler

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72264/#review220086 ---

Re: Review Request 72264: HIVE-23052: Optimize lock enqueueing in TxnHandler

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72264/#review220087 --- LGTM, however there are some issues to consider.

Re: Review Request 72436: Locks: Implement zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72436/ --- (Updated April 27, 2020, 11:24 a.m.) Review request for hive, Marton Bod and

Re: Review Request 72436: Locks: Implement zero-wait readers

> On April 27, 2020, 10:03 a.m., Marton Bod wrote: > > LGMT, just few questions Marton, thank you for the review! > On April 27, 2020, 10:03 a.m., Marton Bod wrote: > > ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java > > Lines 2553 (patched) > >

Review Request 72436: Locks: Implement zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72436/ --- Review request for hive, Marton Bod and Peter Vary. Bugs: HIVE-23293

Re: Review Request 72436: Locks: Implement zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72436/ --- (Updated April 28, 2020, 4:19 p.m.) Review request for hive, Marton Bod and

Re: Review Request 72436: Locks: Implement zero-wait readers

> On April 28, 2020, 9:43 a.m., Peter Vary wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Lines 4353 (patched) > > > > > > why is this move?

Re: Review Request 72436: Locks: Implement zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72436/ --- (Updated April 28, 2020, 4:23 p.m.) Review request for hive, Marton Bod and

Re: Review Request 72436: Locks: Implement zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72436/ --- (Updated April 28, 2020, 5:59 p.m.) Review request for hive, Marton Bod and

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

> On April 3, 2020, 11:28 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java > > Lines 414 (patched) > > > >

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

> On April 3, 2020, 11:28 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-tools/metastore-benchmarks/src/main/java/org/apache/hadoop/hive/metastore/tools/HMSBenchmarks.java > > Lines 414 (patched) > > > >

Re: Review Request 72388: HIVE-23048 Use sequences for TXN_ID generation

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72388/#review220483 ---

Re: Review Request 72387: Locks: Add new lock implementations for always zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72387/ --- (Updated April 22, 2020, 3:27 p.m.) Review request for hive, Marton Bod and

Re: Review Request 72387: Locks: Add new lock implementations for always zero-wait readers

> On April 22, 2020, 3:22 p.m., Peter Vary wrote: > > ql/src/java/org/apache/hadoop/hive/ql/ValidTxnManager.java > > Line 113 (original), 112 (patched) > > > > > > Isn't reference level check is risky here? good

Re: Review Request 72378: HIVE-23201: Improve logging in locking

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72378/#review220456 --- Ship it! Ship It! - Denys Kuzmenko On April 23, 2020, 8:56

Re: Review Request 72378: HIVE-23201: Improve logging in locking

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72378/#review220419 --- Looks good, however I don't really like change with

Re: Review Request 72387: Locks: Add new lock implementations for always zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72387/ --- (Updated April 22, 2020, 2:29 p.m.) Review request for hive, Marton Bod and

Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72470/ --- Review request for hive, Marton Bod, Peter Varga, and Peter Vary. Bugs:

Re: Review Request 72469: HIVE-23325: Clean up cleanup tasks for TxnHandler/CompactionTxnHandler

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72469/#review220594 --- LGTM , just few questions

Re: Review Request 72469: HIVE-23325: Clean up cleanup tasks for TxnHandler/CompactionTxnHandler

> On May 4, 2020, 3:21 p.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/AcidCompactionHistoryService.java > > Line 31 (original), 32 (patched) > > >

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72470/ --- (Updated May 4, 2020, 1:32 p.m.) Review request for hive, Marton Bod, Peter

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72465/#review220593 ---

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72470/ --- (Updated May 4, 2020, 4:33 p.m.) Review request for hive, Marton Bod, Peter

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

> On May 4, 2020, 3:21 p.m., Marton Bod wrote: > > Looks good to me, just a couple of questions. Thank you for the review! > On May 4, 2020, 3:21 p.m., Marton Bod wrote: > > ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java > > Lines 2226 (patched) > >

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

> On May 4, 2020, 3:21 p.m., Marton Bod wrote: > > ql/src/test/org/apache/hadoop/hive/ql/lockmgr/TestDbTxnManager2.java > > Lines 2231 (patched) > > > > > > nit: this appears in a few places, might make sense to

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

> On May 5, 2020, 7:20 a.m., Peter Varga wrote: > > ql/src/java/org/apache/hadoop/hive/ql/Context.java > > Lines 1198 (patched) > > > > > > Do you need a separate field for that, you could just check the > >

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72470/ --- (Updated May 5, 2020, 8:33 a.m.) Review request for hive, Marton Bod, Peter

Re: Review Request 72436: Locks: Implement zero-wait readers

> On April 29, 2020, 9:25 a.m., Marton Bod wrote: > > ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbLockManager.java > > Lines 145 (patched) > > > > > > do we want to handle the case here when lockstate was

Re: Review Request 72436: Locks: Implement zero-wait readers

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72436/ --- (Updated April 29, 2020, 10:18 a.m.) Review request for hive, Marton Bod and

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

> On May 18, 2020, 7:33 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnStatus.java > > Lines 32 (patched) > > > > > > Could you please

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72465/#review220802 --- Thank you for the patch, Peter! LGTM 1 comment on static import

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72112/#review220804 --- LGTM, some minor comments.

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72112/#review220818 ---

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

> On May 19, 2020, 7:28 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-tools/tools-common/src/main/java/org/apache/hadoop/hive/metastore/tools/Util.java > > Line 24 (original), 26 (patched) > > > > > >

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72112/#review220820 --- Ship it! Ship It! - Denys Kuzmenko On May 18, 2020, 4:59

Review Request 72528: ValidTxnManager doesn't consider txns opened and committed between snapshot generation and locking when evaluating ValidTxnListState

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72528/ --- Review request for hive, Peter Varga and Peter Vary. Bugs: HIVE-23503

Re: Review Request 72112: HIVE-22869 - Add locking benchmark to metastore-tools/metastore-benchmarks

> On May 19, 2020, 7:31 a.m., Denys Kuzmenko wrote: > > Ship It! Zoli, I think you mentioned that hms-benchmark doesn't support running tests in multiple threads. Have you tried: -T, --threads= number of concurrent threads - Denys

Re: Review Request 72481: HIVE-23234: Optimize TxnHandler::allocateTableWriteIds

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72481/#review220758 --- LGTM, some minor comments

Re: Review Request 72481: HIVE-23234: Optimize TxnHandler::allocateTableWriteIds

> On May 8, 2020, 10:23 a.m., Peter Vary wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Line 2067 (original), 2057 (patched) > > > > > > Why

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72470/ --- (Updated May 7, 2020, 1:21 p.m.) Review request for hive, Marton Bod, Peter

Re: Review Request 72472: HIVE-23318: TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis

> On May 6, 2020, 4:54 a.m., Jesús Camacho Rodríguez wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java > > Line 3153 (original), 3153 (patched) > > > > > > Unrelated to this patch but related to

Review Request 72249: Rewrite checkLock inner select with JOIN operator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72249/ --- Review request for hive, Peter Vary and Rajesh Balamohan. Bugs: HIVE-22888

Re: Review Request 72246: HIVE-23045: Zookeeper SSL/TLS support

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72246/#review22 --- LGTM, small remarks

Re: Review Request 72249: Rewrite checkLock inner select with JOIN operator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72249/ --- (Updated March 19, 2020, 12:25 p.m.) Review request for hive, Peter Vary and

Re: Review Request 72249: Rewrite checkLock inner select with JOIN operator

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72249/ --- (Updated March 19, 2020, 12:23 p.m.) Review request for hive, Peter Vary and

Re: Review Request 72472: HIVE-23318: TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72472/#review220656 --- Ship it! Ship It! - Denys Kuzmenko On May 5, 2020, 8:58

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

> On May 4, 2020, 3 p.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnUtils.java > > Lines 400 (patched) > > > > > > Is is a generic txn

Re: Review Request 72469: HIVE-23325: Clean up cleanup tasks for TxnHandler/CompactionTxnHandler

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72469/#review220657 --- Ship it! Ship It! - Denys Kuzmenko On May 5, 2020, 7:42

Re: Review Request 72472: HIVE-23318: TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72472/#review220652 --- LGTM, minor comments

Re: Review Request 72472: HIVE-23318: TxnHandler need not delete from MATERIALIZATION_REBUILD_LOCKS on need basis

> On May 6, 2020, 4:54 a.m., Jesús Camacho Rodríguez wrote: > > ql/src/java/org/apache/hadoop/hive/ql/io/AcidUtils.java > > Line 3153 (original), 3153 (patched) > > > > > > Unrelated to this patch but related to

Re: Review Request 72470: ACID: Concurrent MERGE INSERT operations produce duplicates

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72470/ --- (Updated May 7, 2020, 2:49 p.m.) Review request for hive, Marton Bod, Peter

Re: Review Request 72528: ValidTxnManager doesn't consider txns opened and committed between snapshot generation and locking when evaluating ValidTxnListState

> On May 20, 2020, 3:16 p.m., Peter Varga wrote: > > ql/src/java/org/apache/hadoop/hive/ql/Driver.java > > Lines 686 (patched) > > > > > > I have concerns here, but I am not sure if they are well founded or > >

Re: Review Request 72528: ValidTxnManager doesn't consider txns opened and committed between snapshot generation and locking when evaluating ValidTxnListState

> On May 20, 2020, 3:16 p.m., Peter Varga wrote: > > ql/src/java/org/apache/hadoop/hive/ql/Driver.java > > Lines 686 (patched) > > > > > > I have concerns here, but I am not sure if they are well founded or > >

Re: Review Request 72528: ValidTxnManager doesn't consider txns opened and committed between snapshot generation and locking when evaluating ValidTxnListState

> On May 20, 2020, 3:16 p.m., Peter Varga wrote: > > ql/src/java/org/apache/hadoop/hive/ql/Driver.java > > Lines 686 (patched) > > > > > > I have concerns here, but I am not sure if they are well founded or > >

Re: Review Request 72488: HIVE-23413: New config to skip all locks

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72488/#review220947 --- common/src/java/org/apache/hadoop/hive/conf/HiveConf.java Lines

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

> On May 18, 2020, 7:33 a.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnStatus.java > > Lines 32 (patched) > > > > > > Could you please

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72465/#review220950 --- Ship it! Ship It! - Denys Kuzmenko On May 20, 2020, 12:24

Re: Review Request 72465: HIVE-23340 TxnHandler cleanup

- Denys --- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72465/#review220802 --- On May 20, 2020, 12:24 p.m., Peter Varga wrote: > >

Re: Review Request 72528: ValidTxnManager doesn't consider txns opened and committed between snapshot generation and locking when evaluating ValidTxnListState

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72528/ --- (Updated June 9, 2020, 8:52 a.m.) Review request for hive, Jesús Camacho

Re: Review Request 73380: HIVE-21075 Metastore: Drop partition performance downgrade with Postgres DB

--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/73380/#review223064 --- We have switched to GitHub PRs nowadays. - Denys Kuzmenko On

<    1   2