> On Feb. 13, 2020, 3:08 p.m., Denys Kuzmenko wrote: > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > > Lines 4546 (patched) > > <https://reviews.apache.org/r/72129/diff/2/?file=2211214#file2211214line4594> > > > > Hi Rajesh, could you please explain, what is the reason of doing > > partition filtering on HMS side, not backend db? > > Rajesh Balamohan wrote: > By adding all the partition details, the query can become large and has > the issue of overflowing in the case of oracle (i,e have to batch with 1000 > entries). Also, its incurs parsing in sql server side, as it is executed as > Statement. Given that we have additional filter now, it would make it a lot > simpler to do this in client side. This was pointed out in the JIRA by Gopal.
Can we rewrite the query with JOIN operator? Somethinkg like: https://issues.apache.org/jira/browse/HIVE-22888 - Denys ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/72129/#review219575 ----------------------------------------------------------- On Feb. 14, 2020, 1:27 a.m., Rajesh Balamohan wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/72129/ > ----------------------------------------------------------- > > (Updated Feb. 14, 2020, 1:27 a.m.) > > > Review request for hive, Gopal V, Peter Vary, and Zoltan Chovan. > > > Repository: hive-git > > > Description > ------- > > - Main change is in TxnHandler::checkLock. > - When all incoming requests are SHARED_READ, we can add a condition in > the query to retrieve only relevant rows. This avoids significant number of > rows fetched in the form of "SHARED_READ + ACQUIRED". There is a corner > condition of "SHARED_WRITE --> SHARED_READ::ACQUIRED", which is misleading in > the jumpttable. This condition can be optimised later. > - Also, removed the "HL_PARTITION IN" clause which could potentially > overflow for oracle. Partition details can be filtered out, if the earlier > query actually returned any rows. > - Rest of the changes, are related to refactoring > "TxnHandler::enqueueLockWithRetry" to reduce lock scope. > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/lockmgr/DbLockManager.java a8b9653411 > > standalone-metastore/metastore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/TxnHandler.java > f53aebe4ad > > > Diff: https://reviews.apache.org/r/72129/diff/3/ > > > Testing > ------- > > > File Attachments > ---------------- > > HIVE-22850.5.patch > > https://reviews.apache.org/media/uploaded/files/2020/02/13/74ec6cbd-c552-4d46-b5a6-e2fa6da41bdc__HIVE-22850.5.patch > > > Thanks, > > Rajesh Balamohan > >
