This is an automated email from the ASF dual-hosted git repository.
bridgetb pushed a commit to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/drill.git
The following commit(s) were added to refs/heads/gh-pages by this push:
new a79b24f monitoring metrics doc updates
a79b24f is described below
commit a79b24f83b80a21fbf113423147e223857025ca9
Author: Bridget Bevens <[email protected]>
AuthorDate: Fri Nov 9 20:18:21 2018 -0800
monitoring metrics doc updates
---
_docs/performance-tuning/070-monitoring-metrics.md | 93 +++++++++++++++++-----
1 file changed, 74 insertions(+), 19 deletions(-)
diff --git a/_docs/performance-tuning/070-monitoring-metrics.md
b/_docs/performance-tuning/070-monitoring-metrics.md
index d53113b..5e2b374 100644
--- a/_docs/performance-tuning/070-monitoring-metrics.md
+++ b/_docs/performance-tuning/070-monitoring-metrics.md
@@ -1,10 +1,25 @@
---
title: "Monitoring Metrics"
-date: 2016-07-14 18:41:09 UTC
+date: 2018-11-10
parent: "Performance Tuning"
---
-As of Drill 1.7, Drill uses JMX ([Java Management
Extensions](https://docs.oracle.com/javase/tutorial/jmx/)) to monitor queries
at runtime. JMX provides the architecture to dynamically manage and monitor
applications. JMX collects Drill system-level metrics that you can access
through the Metrics tab in the Drill Web Console or a remote JMX monitoring
tool, such as JConsole or the VisualVM + MBeans plugin. The Web Console Metrics
tab contains the collected metrics as tables, counters, hi [...]
+The Metrics page in the Drill Web UI
(http(s)://<drillbit-ip-address>:8047/metrics) lists JVM and operating system
metrics. You can use these metrics to debug the state of the cluster. The
Drill-specific metrics are prepended with `drill`, for example
`drill.fragments.running`. The other metrics relate specifically to the JVM.
+
+Drill uses JMX ([Java Management
Extensions](https://docs.oracle.com/javase/tutorial/jmx/)) to monitor queries
at runtime. JMX provides the architecture to dynamically manage and monitor
applications. JMX collects Drill system-level metrics that you can see in the
Metrics tab in the Drill Web UI or a remote JMX monitoring tool, such as
JConsole or the VisualVM + MBeans plugin.
+
+Metrics collected by JMX are divided into the following categories on the
Metrics page in the Drill Web UI:
+
+* **Gauges**
+A gauge is an instantaneous measure of a value. See
[Gauges]({{site.baseurl}}/docs/monitoring-metrics/#gauges).
+* **Counters**
+A counter is a snapshot of the count of metrics at a particular point in time.
(A gauge for an
[AtomicLong](https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/atomic/AtomicLong.html)
instance.) See [Counters]({{site.baseurl}}/docs/monitoring-metrics/#counters).
+* **Histograms**
+A histogram measures the statistical distribution of values in a stream of
data.
+* **Meters**
+A meter measures the rate of events over time, for example requests per
second.
+* **Timers**
+A timer measures the rate that a particular piece of code is called and the
distribution of its duration.
## Remote Monitoring
You can enable the remote JMX Java feature to monitor a specific JVM from a
remote location. You can enable remote JMX with or without authentication. See
the [Java
documentation](http://docs.oracle.com/javase/7/docs/technotes/guides/management/agent.html).
@@ -18,25 +33,65 @@ JMX metric collection is enabled, by default. You can
disable the metrics option
In `$DRILL_HOME/conf/drill-env.sh`, set the `drill.metrics.jmx.enabled` option
to false through the `DRILLBIT_JAVA_OPTS` variable. Add the variable in
`drill-env.sh` if it does not exist:
- export DRILLBIT_JAVA_OPTS="$DRILLBIT_JAVA_OPTS
-Ddrill.metrics.jmx.enabled=false"
+ export DRILLBIT_JAVA_OPTS="$DRILLBIT_JAVA_OPTS
-Ddrill.metrics.jmx.enabled=false"
+
+
+## Gauges
+
+The following table lists the Drill-specific metrics in the Gauges section of
the Metrics page:
+
+| Metric | Description
[...]
+|--------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
[...]
+| blocked.count | The number of threads that are
blocked because they are waiting on a monitor lock. This metric is useful for
debugging Drill issues.
[...]
+| drill.fragments.running | The number of query fragments
currently running in the Drillbit.
[...]
+| drill.allocator.root.used | The amount of memory (in bytes) used
by the internal memory allocator.
[...]
+| drill.allocator.root.peak | The peak amount of memory (in bytes)
used by the internal memory allocator.
[...]
+| drill.allocator.rpc.bit.control.peak | The maximum amount of bytes used
across all outgoing and incoming control connections for this Drillbit at
this moment.
[...]
+| drill.allocator.rpc.bit.control.used | The total number of bytes currently
used across all outgoing and incoming control connections for this Drillbit.
[...]
+| drill.allocator.rpc.bit.data.peak | The maximum amount of memory used
between all outgoing and incoming data connections for this Drillbit up to
this moment.
[...]
+| drill.allocator.rpc.bit.data.used | The total amount of memory used
between all outgoing and incoming data connections tor this Drillbit.
[...]
+| drill.allocator.rpc.bit.user.peak | The maximum amount of memory used
across all incoming Drill client connections to this Drillbit up to this
moment.
[...]
+| drill.allocator.rpc.bit.user.used | The total amount of memory used
across all incoming Drill client connections to this Drillbit.
[...]
+| drill.allocator.huge.size | Total size in bytes of huge (greater
than 16MB) direct buffers allocated until now
[...]
+| drill.allocator.huge.count | Number of allocations done for
direct buffer of size greater than 16MB. Each of these allocation happens
from OS which comes with an overhead rather from Netty's buffer pool.
[...]
+| drill.allocator.normal.count | Number of allocations done for
direct buffer of size less than equal to 16MB. Each of these allocation
happens from Netty's buffer pool. This counter is only updated in debug
environment when asserts are enabled to avoid overhead for each allocation
during normal execution.
[...]
+| drill.allocator.normal.size | Total size in bytes of normal (less
than and equal to 16MB) direct buffers allocated until now. This counter is
only updated in debug environment when asserts are enabled to avoid overhead
for each allocation during normal execution.
[...]
+| drill.allocator.normal.size | Total size in bytes of normal (less
than and equal to 16MB) direct buffers allocated until now. This counter is
only updated in debug environment when asserts are enabled to avoid overhead
for each allocation during normal execution.
[...]
+| heap.used | The amount of heap memory (in
bytes) used by the JVM.
+|
[...]
+|
+| non-heap.used | The amount of non-heap memory (in
bytes) used by the JVM.
[...]
+| fd.usage | The ratio of used file descriptors
to total file descriptors on *nix systems.
[...]
+| direct.used | The amount of direct memory (in
bytes) used by the JVM. This metric is useful for debugging Drill issues.
[...]
+| runnable.count | The number of threads executing an
action in the JVM. This metric is useful for debugging Drill issues.
[...]
+| waiting.count | The number of threads waiting to
execute. Typically, threads waiting on other threads to perform an action.
This metric is useful for debugging Drill issues.
[...]
+| load.avg | Returns the "recent cpu usage" for
the Drillbit process. This value is a double in the [0.0,1.0] interval. A
value of 0.0 means that none of the CPUs were running threads from the
Drillbit process during the recent period of time observed, while a value of
1.0 means that all CPUs were actively running threads from the Drillbit
process 100% of the time during the recent period being observed. Threads
from the Drillbit process includes t [...]
+| uptime | Total uptime of Drillbit JVM in
miliseconds. See
[getUptime()](https://docs.oracle.com/javase/7/docs/api/java/lang/management/RuntimeMXBean.html#getUptime()).
[...]
+
+
+## Counters
+
+The following table lists the Drill-specific metrics in the Counters section
of the Metrics page:
+
+| Metric | Description
|
+|-------------------------|-----------------------------------------------------------------------------------------------|
+| drill.queries.running | The number of queries running for which this
drillbit is the Foreman. |
+| drill.queries.completed | The number of queries completed, canceled, or
failed for which this drillbit was the Foreman. |
+
+
+## Histograms
+
+The following table lists the Drill-specific metrics in the Histograms section
of the Metrics page:
-The following table lists the predefined Drill system-level metrics that you
can view through a JMX monitoring tool or the Drill Web Console:
+| Metric | Description
|
+|-----------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| drill.allocator.huge.hist | Displays the distribution of allocation of
huge buffers up to the current time. Like count, it specifies number of huge
buffer allocations completed so far. Max/Min specifies maximum/minimum size
in bytes of the huge buffer allocated. Mean and other percentiles show the
distribution of the huge buffer allocation size in bytes. |
+| drill.allocator.normal.hist | Displays the distribution of allocation of the
normal size buffers up to the current time. Like count, it specifies the
number of normal buffer allocations completed so far. Max/Min specifies
maximum/minimum size in bytes of the normal buffer allocated. Mean and other
percentiles show the distribution of normal buffer allocation size in bytes. |
-| Metric | Description
|
-|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| drill.queries.running | The number of queries running for which this
drillbit is the Foreman.
|
-| drill.queries.completed | The number of queries completed, canceled, or
failed for which this drillbit was the Foreman.
|
-| drill.fragments.running | The number of query fragments currently
running in the drillbit.
|
-| drill.allocator.root.used | The amount of memory (in bytes) used by the
internal memory allocator.
|
-| drill.allocator.root.peak | The peak amount of memory (in bytes) used by
the internal memory allocator.
|
-| heap.used | The amount of heap memory (in bytes) used by
the JVM.
|
-| non-heap.used | The amount of non-heap memory (in bytes) used
by the JVM.
|
-| count | The number of live threads, including daemon
and non-daemon threads.
|
-| fd.usage | The ratio of used file descriptors to total
file descriptors on *nix systems.
|
-| direct.used | The amount of direct memory (in bytes) used by
the JVM. This metric is useful for debugging Drill issues.
|
-| runnable.count | The number of threads executing an action in
the JVM. This metric is useful for debugging Drill issues.
|
-| waiting.count | The number of threads waiting to execute.
Typically, threads waiting on other threads to perform an action. This metric
is useful for debugging Drill issues. |
-| blocked.count | The number of threads that are blocked because
they are waiting on a monitor lock. This metric is useful for debugging Drill
issues. |
+## Meters
+No Drill-specific meters.
+## Timers
+No Drill-specific timers.
\ No newline at end of file
