[
https://issues.apache.org/jira/browse/HIVE-23052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Bod updated HIVE-23052:
------------------------------
Description:
* Reduce scope of next_lock_id select-for-update by moving the txn_component
inserts before the S4U + inserting the hive_locks entries before the S4U (first
with a temp ID, which will be replaced later in a single update). This helps
decrease the overall time that the next_lock_id table is locked, thereby
increasing concurrency
* Insert txn_components in a batch instead of one-by-one
* Increment next_lock_id and update hive_locks table in a single batch
statement
was:
TxnHandler::enqueueLockWithRetry uses select for update (S4U) to retrieve the
next lock id from the metastore. This locks the NEXT_LOCK_ID table for
concurrent lock requests until the transaction has been committed.
Since this creates contention, we should reduce the scope of this S4U, thereby
locking the table for only as long as necessary. Currently, within the same
txn, after the initial S4U, we also insert the transaction components
(TXN_COMPONENTS table) before committing, even though they are independent and
do not make use of the lock id. So moving these txn_component inserts before
the S4U would decrease the time the next_lock_id table is locked and thus
increase concurrency.
> Optimize lock enqueueing in TxnHandler
> --------------------------------------
>
> Key: HIVE-23052
> URL: https://issues.apache.org/jira/browse/HIVE-23052
> Project: Hive
> Issue Type: Improvement
> Reporter: Marton Bod
> Assignee: Marton Bod
> Priority: Major
> Attachments: HIVE-23052.1.patch, HIVE-23052.2.patch,
> HIVE-23052.3.patch
>
>
> * Reduce scope of next_lock_id select-for-update by moving the txn_component
> inserts before the S4U + inserting the hive_locks entries before the S4U
> (first with a temp ID, which will be replaced later in a single update). This
> helps decrease the overall time that the next_lock_id table is locked,
> thereby increasing concurrency
> * Insert txn_components in a batch instead of one-by-one
> * Increment next_lock_id and update hive_locks table in a single batch
> statement
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
