Oliver Draese created HIVE-21204:
------------------------------------
Summary: Instrumentation for read/write locks in LLAP
Key: HIVE-21204
URL: https://issues.apache.org/jira/browse/HIVE-21204
Project: Hive
Issue Type: Improvement
Components: llap
Reporter: Oliver Draese
Assignee: Oliver Draese
LLAP has several R/W locks for serialization of updates to query tracker, file
data, ....
Instrumentation is added to monitor the
* total amount of R/W locks within a particular category
* average + max wait/suspension time to get the R/W lock
A category includes all lock instances for particular areas (i.e. category is
FileData and all R/W locks that are used in FileData instances are accounted
within the one category).
The monitoring/accounting is done via Hadoop Metrics 2, making them accessible
via JMX. In addition, a new "locking" GET endpoint is added to the LLAP
daemon's REST interface. It produces output like the following example:
{{{}}
{{ "statsCollection": "enabled",}}
{{ "lockStats": [}}
{{ {}}{{ "type": "R/W Lock Stats",}}
{{ "label": "FileData",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ }}}
{{ },}}
{{ { "}}{{type": "R/W Lock Stats",}}
{{ "label": "QueryTracker",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ }}}
{{ } }}{{]}}
{{ }}}
{{}}}
To avoid the overhead of lock instrumentation, lock metrics collection is
disabled by default and can be enabled via the following configuration
parameter:
{{hive.llap.lockmetrics.collect = true}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
