[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
slim bouguerra updated HIVE-21204:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch,
> HIVE-21204.6.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.6.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch,
> HIVE-21204.6.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.5.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.5.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.5.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.5.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.5.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.5.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.5.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch, HIVE-21204.5.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.2.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.1.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: (was: HIVE-21204.3.patch)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.4.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.4.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.3.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.1.patch, HIVE-21204.2.patch,
> HIVE-21204.3.patch, HIVE-21204.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.2.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.1.patch, HIVE-21204.2.patch, HIVE-21204.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.1.patch
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.1.patch, HIVE-21204.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> }
> {{ } }}{{]}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Description:
LLAP has several R/W locks for serialization of updates to query tracker, file
data,
Instrumentation is added to monitor the
* total amount of R/W locks within a particular category
* average + max wait/suspension time to get the R/W lock
A category includes all lock instances for particular areas (i.e. category is
FileData and all R/W locks that are used in FileData instances are accounted
within the one category).
The monitoring/accounting is done via Hadoop Metrics 2, making them accessible
via JMX. In addition, a new "locking" GET endpoint is added to the LLAP
daemon's REST interface. It produces output like the following example:
{
{{ "statsCollection": "enabled",}}
{{ "lockStats": [}}
{{ {}}{{ "type": "R/W Lock Stats",}}
{{ "label": "FileData",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
}
{{ },}}
{{ { "}}{{type": "R/W Lock Stats",}}
{{ "label": "QueryTracker",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
}
{{ } }}{{]}}
{{}}}
To avoid the overhead of lock instrumentation, lock metrics collection is
disabled by default and can be enabled via the following configuration
parameter:
{{hive.llap.lockmetrics.collect = true}}
was:
LLAP has several R/W locks for serialization of updates to query tracker, file
data,
Instrumentation is added to monitor the
* total amount of R/W locks within a particular category
* average + max wait/suspension time to get the R/W lock
A category includes all lock instances for particular areas (i.e. category is
FileData and all R/W locks that are used in FileData instances are accounted
within the one category).
The monitoring/accounting is done via Hadoop Metrics 2, making them accessible
via JMX. In addition, a new "locking" GET endpoint is added to the LLAP
daemon's REST interface. It produces output like the following example:
{{{}}
{{ "statsCollection": "enabled",}}
{{ "lockStats": [}}
{{ {}}{{ "type": "R/W Lock Stats",}}
{{ "label": "FileData",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ }}}
{{ },}}
{{ { "}}{{type": "R/W Lock Stats",}}
{{ "label": "QueryTracker",}}
{{ "totalLockWaitTimeMillis": 0,}}
{{ "readLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ },}}
{{ "writeLock": {}}
{{ "count": 0,}}
{{ "avgWaitTimeNanos": 0,}}
{{ "maxWaitTimeNanos": 0}}
{{ }}}
{{ } }}{{]}}
{{ }}}
{{}}}
To avoid the overhead of lock instrumentation, lock metrics collection is
disabled by default and can be enabled via the following configuration
parameter:
{{hive.llap.lockmetrics.collect = true}}
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{
[jira] [Updated] (HIVE-21204) Instrumentation for read/write locks in LLAP
[
https://issues.apache.org/jira/browse/HIVE-21204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oliver Draese updated HIVE-21204:
-
Attachment: HIVE-21204.patch
Status: Patch Available (was: Open)
> Instrumentation for read/write locks in LLAP
>
>
> Key: HIVE-21204
> URL: https://issues.apache.org/jira/browse/HIVE-21204
> Project: Hive
> Issue Type: Improvement
> Components: llap
>Reporter: Oliver Draese
>Assignee: Oliver Draese
>Priority: Major
> Attachments: HIVE-21204.patch
>
>
> LLAP has several R/W locks for serialization of updates to query tracker,
> file data,
> Instrumentation is added to monitor the
> * total amount of R/W locks within a particular category
> * average + max wait/suspension time to get the R/W lock
> A category includes all lock instances for particular areas (i.e. category is
> FileData and all R/W locks that are used in FileData instances are accounted
> within the one category).
> The monitoring/accounting is done via Hadoop Metrics 2, making them
> accessible via JMX. In addition, a new "locking" GET endpoint is added to the
> LLAP daemon's REST interface. It produces output like the following example:
> {{{}}
> {{ "statsCollection": "enabled",}}
> {{ "lockStats": [}}
> {{ {}}{{ "type": "R/W Lock Stats",}}
> {{ "label": "FileData",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ }}}
> {{ },}}
> {{ { "}}{{type": "R/W Lock Stats",}}
> {{ "label": "QueryTracker",}}
> {{ "totalLockWaitTimeMillis": 0,}}
> {{ "readLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ },}}
> {{ "writeLock": {}}
> {{ "count": 0,}}
> {{ "avgWaitTimeNanos": 0,}}
> {{ "maxWaitTimeNanos": 0}}
> {{ }}}
> {{ } }}{{]}}
> {{ }}}
> {{}}}
> To avoid the overhead of lock instrumentation, lock metrics collection is
> disabled by default and can be enabled via the following configuration
> parameter:
> {{hive.llap.lockmetrics.collect = true}}
>
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
