Joe McDonnell created IMPALA-12021:
--------------------------------------
Summary: Use fair scheduling policy for
CatalogOpExecutor::metastoreDdlLock_
Key: IMPALA-12021
URL: https://issues.apache.org/jira/browse/IMPALA-12021
Project: IMPALA
Issue Type: Improvement
Components: Catalog
Affects Versions: Impala 4.3.0
Reporter: Joe McDonnell
CatalogOpExecutor's metastoreDdlLock_ is a lock inside the Catalog that
synchronizes operations to create/drop tables, databases, etc. It currently
uses a ReentrantLock, and ReentrantLock has an option to use fair scheduling:
[https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/ReentrantLock.html#ReentrantLock(boolean)]
Here is a description of the fairness option:
{noformat}
The constructor for this class accepts an optional fairness parameter. When set
true, under contention, locks favor granting access to the longest-waiting
thread. Otherwise this lock does not guarantee any particular access order.
Programs using fair locks accessed by many threads may display lower overall
throughput (i.e., are slower; often much slower) than those using the default
setting, but have smaller variances in times to obtain locks and guarantee lack
of starvation.{noformat}
It seems like this lock does not have high throughput (there are only so many
create / drops per second). If the create/drops are taking noticeable time
(seconds), then fair scheduling would be desirable to reduce the tail latency.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
